Jul 30 06:43:57.247026 ip-10-0-143-185 systemd[1]: Starting Kubernetes Kubelet... Jul 30 06:43:57.707518 ip-10-0-143-185 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 06:43:57.707518 ip-10-0-143-185 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 30 06:43:57.707518 ip-10-0-143-185 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 06:43:57.707518 ip-10-0-143-185 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 30 06:43:57.707518 ip-10-0-143-185 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 30 06:43:57.708675 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.708591 2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 30 06:43:57.713875 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713857 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 06:43:57.713875 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713875 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713879 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713882 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713885 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713889 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713891 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713894 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713900 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713903 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713906 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713916 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713919 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713922 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713925 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713927 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713930 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713933 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713935 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713938 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 06:43:57.713940 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713940 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713944 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713948 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713951 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713954 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713956 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713959 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713962 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713965 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713968 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713971 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713973 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713976 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713979 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713983 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713987 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713990 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713992 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713995 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 06:43:57.714408 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.713998 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714001 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714004 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714007 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714010 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714012 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714015 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714017 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714020 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714022 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714025 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714027 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714030 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714032 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714034 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714038 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714041 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714044 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714047 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714049 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 06:43:57.714883 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714052 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714054 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714057 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714061 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714063 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714066 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714069 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714071 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714073 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714076 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714079 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714081 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714092 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714095 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714097 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714100 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714102 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714105 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714108 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714110 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 06:43:57.715402 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714113 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714115 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714118 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714120 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714123 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714125 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714128 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714132 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714135 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714138 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714141 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714143 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714146 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714149 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714151 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714154 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714156 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714159 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714161 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 06:43:57.715904 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.714164 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715912 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715920 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715923 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715926 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715930 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715934 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715937 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715940 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715942 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715945 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715948 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715950 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715953 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715955 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715958 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715961 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715963 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715966 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715968 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 06:43:57.716397 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715971 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715973 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715976 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715978 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715981 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715983 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715985 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715988 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715991 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715993 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715995 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.715999 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716003 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716005 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716009 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716012 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716015 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716017 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716020 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 06:43:57.716905 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716023 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716025 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716028 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716032 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716034 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716036 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716039 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716041 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716044 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716046 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716049 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716051 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716053 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716056 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716058 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716061 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716063 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716068 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716071 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716074 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 06:43:57.717391 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716076 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716079 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716081 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716083 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716086 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716088 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716091 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716094 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716097 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716099 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716102 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716105 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716107 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716112 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716114 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716117 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716120 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716123 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716126 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 06:43:57.717890 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716128 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716131 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716133 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716136 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716138 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716141 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716143 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716146 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716148 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716151 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716154 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716156 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716159 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716161 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716163 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716166 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716168 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716171 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716173 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 06:43:57.718352 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716176 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716179 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716183 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716591 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716596 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716600 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716603 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716613 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716618 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716622 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716625 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716628 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716631 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716633 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716636 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716639 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716641 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716644 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 06:43:57.718827 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716646 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716649 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716652 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716654 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716657 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716659 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716662 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716665 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716667 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716670 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716672 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716676 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716678 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716682 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716684 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716687 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716689 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716692 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716695 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 06:43:57.719261 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716700 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716704 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716707 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716711 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716713 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716716 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716719 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716721 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716724 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716727 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716729 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716732 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716735 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716738 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716741 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716743 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716746 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716748 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716771 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716775 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 06:43:57.719727 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716778 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716782 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716784 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716787 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716790 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716793 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716795 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716798 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716801 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716805 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716807 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716810 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716813 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716815 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716818 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716820 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716824 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716826 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716829 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716831 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 06:43:57.720235 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716833 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716836 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716839 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716841 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716844 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716846 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716849 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716851 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716853 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716856 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716858 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716861 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716863 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716866 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716868 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716871 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716873 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716876 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716879 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716881 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 06:43:57.720702 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716884 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716886 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716888 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716891 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.716893 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.716961 2575 flags.go:64] FLAG: --address="0.0.0.0" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.716970 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.716977 2575 flags.go:64] FLAG: --anonymous-auth="true" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.716981 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.716986 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.716989 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.716994 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.716998 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717001 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717004 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717008 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717011 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717014 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717017 2575 flags.go:64] FLAG: --cgroup-root="" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717020 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717023 2575 flags.go:64] FLAG: --client-ca-file="" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717026 2575 flags.go:64] FLAG: --cloud-provider="external" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717029 2575 flags.go:64] FLAG: --cluster-dns="[]" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717036 2575 flags.go:64] FLAG: --cluster-domain="" Jul 30 06:43:57.721214 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717039 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717042 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717046 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717049 2575 flags.go:64] FLAG: --container-log-max-files="5" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717053 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717056 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717059 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717062 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717065 2575 flags.go:64] FLAG: --contention-profiling="false" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717068 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717071 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717074 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717077 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717081 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717084 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717087 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717090 2575 flags.go:64] FLAG: --enable-load-reader="false" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717093 2575 flags.go:64] FLAG: --enable-server="true" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717096 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717100 2575 flags.go:64] FLAG: --event-burst="100" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717104 2575 flags.go:64] FLAG: --event-qps="50" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717107 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717110 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717114 2575 flags.go:64] FLAG: --eviction-hard="" Jul 30 06:43:57.721800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717117 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717120 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717128 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717131 2575 flags.go:64] FLAG: --eviction-soft="" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717134 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717137 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717140 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717144 2575 flags.go:64] FLAG: --experimental-mounter-path="" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717147 2575 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717149 2575 flags.go:64] FLAG: --fail-swap-on="true" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717152 2575 flags.go:64] FLAG: --feature-gates="" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717156 2575 flags.go:64] FLAG: --file-check-frequency="20s" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717159 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717162 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717166 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717168 2575 flags.go:64] FLAG: --healthz-port="10248" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717171 2575 flags.go:64] FLAG: --help="false" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717174 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-143-185.ec2.internal" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717177 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717180 2575 flags.go:64] FLAG: --http-check-frequency="20s" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717183 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717186 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717189 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717192 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 30 06:43:57.722362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717195 2575 flags.go:64] FLAG: --image-service-endpoint="" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717197 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717200 2575 flags.go:64] FLAG: --kube-api-burst="100" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717203 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717206 2575 flags.go:64] FLAG: --kube-api-qps="50" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717209 2575 flags.go:64] FLAG: --kube-reserved="" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717212 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717214 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717217 2575 flags.go:64] FLAG: --kubelet-cgroups="" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717220 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717224 2575 flags.go:64] FLAG: --lock-file="" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717227 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717230 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717236 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717241 2575 flags.go:64] FLAG: --log-json-split-stream="false" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717244 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717247 2575 flags.go:64] FLAG: --log-text-split-stream="false" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717250 2575 flags.go:64] FLAG: --logging-format="text" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717253 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717256 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717259 2575 flags.go:64] FLAG: --manifest-url="" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717262 2575 flags.go:64] FLAG: --manifest-url-header="" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717266 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717269 2575 flags.go:64] FLAG: --max-open-files="1000000" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717273 2575 flags.go:64] FLAG: --max-pods="110" Jul 30 06:43:57.723023 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717276 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717279 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717282 2575 flags.go:64] FLAG: --memory-manager-policy="None" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717285 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717288 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717291 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717293 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717300 2575 flags.go:64] FLAG: --node-status-max-images="50" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717303 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717306 2575 flags.go:64] FLAG: --oom-score-adj="-999" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717309 2575 flags.go:64] FLAG: --pod-cidr="" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717312 2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717318 2575 flags.go:64] FLAG: --pod-manifest-path="" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717321 2575 flags.go:64] FLAG: --pod-max-pids="-1" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717325 2575 flags.go:64] FLAG: --pods-per-core="0" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717327 2575 flags.go:64] FLAG: --port="10250" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717330 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717333 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bc79689f633798ac" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717338 2575 flags.go:64] FLAG: --qos-reserved="" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717340 2575 flags.go:64] FLAG: --read-only-port="10255" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717347 2575 flags.go:64] FLAG: --register-node="true" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717350 2575 flags.go:64] FLAG: --register-with-taints="" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717353 2575 flags.go:64] FLAG: --registry-burst="10" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717356 2575 flags.go:64] FLAG: --registry-qps="5" Jul 30 06:43:57.723652 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717359 2575 flags.go:64] FLAG: --reserved-cpus="" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717362 2575 flags.go:64] FLAG: --reserved-memory="" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717366 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717369 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717372 2575 flags.go:64] FLAG: --rotate-certificates="false" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717375 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717378 2575 flags.go:64] FLAG: --runonce="false" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717381 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717384 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717387 2575 flags.go:64] FLAG: --seccomp-default="false" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717390 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717393 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717396 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717399 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717402 2575 flags.go:64] FLAG: --storage-driver-password="root" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717405 2575 flags.go:64] FLAG: --storage-driver-secure="false" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717408 2575 flags.go:64] FLAG: --storage-driver-table="stats" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717410 2575 flags.go:64] FLAG: --storage-driver-user="root" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717414 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717416 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717420 2575 flags.go:64] FLAG: --system-cgroups="" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717422 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717428 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717431 2575 flags.go:64] FLAG: --tls-cert-file="" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717434 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 30 06:43:57.724274 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717438 2575 flags.go:64] FLAG: --tls-min-version="" Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717442 2575 flags.go:64] FLAG: --tls-private-key-file="" Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717445 2575 flags.go:64] FLAG: --topology-manager-policy="none" Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717449 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717452 2575 flags.go:64] FLAG: --topology-manager-scope="container" Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717455 2575 flags.go:64] FLAG: --v="2" Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717459 2575 flags.go:64] FLAG: --version="false" Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717463 2575 flags.go:64] FLAG: --vmodule="" Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717466 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.717470 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717605 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717609 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717611 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717614 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717617 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717620 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717623 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717626 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717629 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717631 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717634 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 06:43:57.725067 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717637 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717639 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717642 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717645 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717647 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717650 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717652 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717655 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717657 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717660 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717662 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717665 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717669 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717671 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717675 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717678 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717680 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717683 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717685 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717688 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 06:43:57.725578 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717690 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717694 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717698 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717701 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717703 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717707 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717710 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717712 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717715 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717718 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717720 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717723 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717725 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717728 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717730 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717733 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717735 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717738 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717740 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 06:43:57.726082 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717743 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717747 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717750 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717770 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717775 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717781 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717783 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717788 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717790 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717793 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717795 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717798 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717800 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717803 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717806 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717808 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717811 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717813 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717816 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 06:43:57.726579 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717818 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717821 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717824 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717826 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717829 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717832 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717834 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717837 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717839 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717842 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717844 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717847 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717849 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717852 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717855 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717858 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717861 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717863 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717867 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717869 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 06:43:57.727231 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717875 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 06:43:57.727969 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717878 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 06:43:57.727969 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717881 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 06:43:57.727969 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717883 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 06:43:57.727969 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717886 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 06:43:57.727969 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717888 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 06:43:57.727969 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717891 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 06:43:57.727969 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717894 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 30 06:43:57.727969 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717897 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 06:43:57.727969 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.717900 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 06:43:57.727969 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.718649 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 06:43:57.727969 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.726016 2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 30 06:43:57.727969 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.726031 2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 30 06:43:57.727969 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.726050 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 30 06:43:57.727969 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.726056 2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 30 06:43:57.727969 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726110 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726115 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726118 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726121 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726124 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726127 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726130 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726133 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726136 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726139 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726141 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726144 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726147 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726149 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726152 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726154 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726157 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726159 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726162 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726164 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 06:43:57.728343 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726166 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726169 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726171 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726174 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726176 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726179 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726181 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726184 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726186 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726190 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726193 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726195 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726198 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726200 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726203 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726205 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726208 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726210 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726212 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726215 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 06:43:57.728842 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726217 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726219 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726222 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726226 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726229 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726232 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726235 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726238 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726241 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726243 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726246 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726248 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726251 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726253 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726256 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726258 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726261 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726263 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726266 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 06:43:57.729319 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726268 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726271 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726274 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726277 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726279 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726282 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726286 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726289 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726292 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726294 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726297 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726300 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726302 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726305 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726307 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726310 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726313 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726316 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726319 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 06:43:57.729811 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726322 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726324 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726327 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726330 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726332 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726335 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726337 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726340 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726342 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726344 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726347 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726349 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726352 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726355 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726357 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726360 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726364 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726367 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726369 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 06:43:57.730301 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726372 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 06:43:57.730774 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726374 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 06:43:57.730774 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.726379 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 06:43:57.730774 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726484 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 30 06:43:57.730774 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726488 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 30 06:43:57.730774 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726491 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 30 06:43:57.730774 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726493 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 30 06:43:57.730774 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726496 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 30 06:43:57.730774 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726499 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 30 06:43:57.730774 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726501 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 30 06:43:57.730774 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726504 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 30 06:43:57.730774 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726507 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 30 06:43:57.730774 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726509 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 30 06:43:57.730774 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726511 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 30 06:43:57.730774 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726514 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 30 06:43:57.730774 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726516 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726518 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726521 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726523 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726526 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726529 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726533 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726535 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726538 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726541 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726544 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726546 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726549 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726552 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726555 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726557 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726560 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726562 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726565 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 30 06:43:57.731141 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726567 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726569 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726572 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726574 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726577 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726579 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726582 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726584 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726587 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726589 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726592 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726595 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726599 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726601 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726604 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726606 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726608 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726611 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726613 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 30 06:43:57.731618 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726615 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726618 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726620 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726623 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726625 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726627 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726630 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726633 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726635 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726638 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726640 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726685 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726732 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726737 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726741 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726743 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726746 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726748 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726751 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726773 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 30 06:43:57.732172 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726777 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726782 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.726786 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727419 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727429 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727434 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727439 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727444 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727448 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727453 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727458 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727462 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727467 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727472 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727508 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727520 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727525 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727530 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727534 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727539 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 30 06:43:57.732659 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727543 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 30 06:43:57.733162 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727547 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 30 06:43:57.733162 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727552 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 30 06:43:57.733162 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727556 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 30 06:43:57.733162 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727560 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 30 06:43:57.733162 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727564 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 30 06:43:57.733162 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727568 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 30 06:43:57.733162 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727576 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 30 06:43:57.733162 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:57.727580 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 30 06:43:57.733162 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.727589 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 30 06:43:57.733162 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.728180 2575 server.go:962] "Client rotation is on, will bootstrap in background" Jul 30 06:43:57.733162 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.731474 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 30 06:43:57.733434 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.733240 2575 server.go:1019] "Starting client certificate rotation" Jul 30 06:43:57.733434 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.733341 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 30 06:43:57.733434 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.733373 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 30 06:43:57.736041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.736027 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 30 06:43:57.737381 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.737362 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 30 06:43:57.746056 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.746038 2575 log.go:25] "Validated CRI v1 runtime API" Jul 30 06:43:57.748536 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.748519 2575 log.go:25] "Validated CRI v1 image API" Jul 30 06:43:57.752499 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.752478 2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 30 06:43:57.757352 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.757330 2575 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b1351165-0cdb-446e-a208-b8b5e8f2c587:/dev/nvme0n1p3 f92281e6-9807-481e-abd6-235a60c542ac:/dev/nvme0n1p4] Jul 30 06:43:57.757428 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.757349 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 30 06:43:57.763045 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.762929 2575 manager.go:216] Machine: {Timestamp:2026-07-30 06:43:57.761580818 +0000 UTC m=+0.378484520 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3090932 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2798dfd501da4fe7a5db81a013500d SystemUUID:ec2798df-d501-da4f-e7a5-db81a013500d BootID:b9a000fb-de4d-4bb6-9078-63fafcc09097 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:20:25:d3:63:a9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:20:25:d3:63:a9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:0b:7e:45:66:6c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 30 06:43:57.763045 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.763038 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 30 06:43:57.763177 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.763116 2575 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 30 06:43:57.764475 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.764457 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 30 06:43:57.766600 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.766576 2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 30 06:43:57.766740 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.766604 2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-185.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 30 06:43:57.768163 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.768152 2575 topology_manager.go:138] "Creating topology manager with none policy" Jul 30 06:43:57.768203 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.768164 2575 container_manager_linux.go:309] "Creating device plugin manager" Jul 30 06:43:57.768203 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.768171 2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 30 06:43:57.769082 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.769071 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 30 06:43:57.770265 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.770255 2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 30 06:43:57.773024 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.773013 2575 state_mem.go:36] "Initialized new in-memory state store" Jul 30 06:43:57.773168 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.773158 2575 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 30 06:43:57.773205 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.773188 2575 kubelet.go:486] "Attempting to sync node with API server" Jul 30 06:43:57.773236 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.773219 2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 30 06:43:57.773236 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.773236 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 30 06:43:57.773319 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.773247 2575 kubelet.go:398] "Adding apiserver pod source" Jul 30 06:43:57.773319 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.773257 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 30 06:43:57.773809 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.773795 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 30 06:43:57.773863 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.773820 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 30 06:43:57.778145 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.778122 2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 30 06:43:57.780443 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.780429 2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 30 06:43:57.780495 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.780460 2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 30 06:43:57.783171 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.783159 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 30 06:43:57.783219 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.783180 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 30 06:43:57.783219 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.783187 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 30 06:43:57.783219 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.783192 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 30 06:43:57.783219 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.783202 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 30 06:43:57.783219 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.783207 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 30 06:43:57.783219 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.783214 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 30 06:43:57.783219 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.783219 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 30 06:43:57.783407 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.783227 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 30 06:43:57.783407 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.783232 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 30 06:43:57.783407 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.783241 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 30 06:43:57.783407 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.783245 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 30 06:43:57.784121 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.784109 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 30 06:43:57.784121 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.784121 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 30 06:43:57.784460 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:57.784435 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 30 06:43:57.784495 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:57.784433 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-185.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 30 06:43:57.787264 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.787252 2575 server.go:1268] "Started kubelet" Jul 30 06:43:57.787386 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.787339 2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 30 06:43:57.787440 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.787346 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 30 06:43:57.787440 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.787425 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 30 06:43:57.788206 ip-10-0-143-185 systemd[1]: Started Kubernetes Kubelet. Jul 30 06:43:57.788846 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.788810 2575 server.go:310] "Adding debug handlers to kubelet server" Jul 30 06:43:57.789657 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.789642 2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 30 06:43:57.793342 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:57.793318 2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 30 06:43:57.794408 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.794393 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-185.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 30 06:43:57.795435 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:57.794341 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-185.ec2.internal.18c6fe3c6d7a8df6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-185.ec2.internal,UID:ip-10-0-143-185.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-185.ec2.internal,},FirstTimestamp:2026-07-30 06:43:57.787221494 +0000 UTC m=+0.404125197,LastTimestamp:2026-07-30 06:43:57.787221494 +0000 UTC m=+0.404125197,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-185.ec2.internal,}" Jul 30 06:43:57.799911 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.799894 2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 30 06:43:57.799911 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.799908 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 30 06:43:57.800055 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:57.799972 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jul 30 06:43:57.800656 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.800638 2575 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 30 06:43:57.800656 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.800659 2575 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 30 06:43:57.800805 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.800643 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 30 06:43:57.800805 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.800747 2575 factory.go:55] Registering systemd factory Jul 30 06:43:57.800805 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.800774 2575 reconstruct.go:97] "Volume reconstruction finished" Jul 30 06:43:57.800805 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.800785 2575 reconciler.go:29] "Reconciler: start to sync state" Jul 30 06:43:57.800805 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.800799 2575 factory.go:223] Registration of the systemd container factory successfully Jul 30 06:43:57.801044 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.801012 2575 factory.go:153] Registering CRI-O factory Jul 30 06:43:57.801044 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.801029 2575 factory.go:223] Registration of the crio container factory successfully Jul 30 06:43:57.801128 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.801102 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 30 06:43:57.801128 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.801125 2575 factory.go:103] Registering Raw factory Jul 30 06:43:57.801212 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.801136 2575 manager.go:1196] Started watching for new ooms in manager Jul 30 06:43:57.801483 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.801472 2575 manager.go:349] Starting recovery of all containers Jul 30 06:43:57.801905 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:57.801884 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 30 06:43:57.802008 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:57.801895 2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-185.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 30 06:43:57.807194 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.807165 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2zrmw" Jul 30 06:43:57.814154 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.813981 2575 manager.go:354] Recovery completed Jul 30 06:43:57.814563 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.814539 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2zrmw" Jul 30 06:43:57.819640 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.819625 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 06:43:57.821510 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.821492 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientMemory" Jul 30 06:43:57.821585 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.821529 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 06:43:57.821585 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.821545 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientPID" Jul 30 06:43:57.822148 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.822133 2575 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 30 06:43:57.822148 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.822148 2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 30 06:43:57.822269 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.822157 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 06:43:57.822269 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.822163 2575 state_mem.go:36] "Initialized new in-memory state store" Jul 30 06:43:57.822866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.822849 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientMemory" Jul 30 06:43:57.822941 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.822881 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 06:43:57.822941 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.822897 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientPID" Jul 30 06:43:57.824779 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.824748 2575 policy_none.go:49] "None policy: Start" Jul 30 06:43:57.824835 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.824782 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 30 06:43:57.824835 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.824793 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 30 06:43:57.826606 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.826596 2575 policy_none.go:47] "Start" Jul 30 06:43:57.862467 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.862448 2575 manager.go:337] "Starting Device Plugin manager" Jul 30 06:43:57.885686 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:57.862491 2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 30 06:43:57.885686 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.862506 2575 server.go:87] "Starting device plugin registration server" Jul 30 06:43:57.885686 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.862803 2575 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 30 06:43:57.885686 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.862816 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 30 06:43:57.885686 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.862917 2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 30 06:43:57.885686 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.862995 2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 30 06:43:57.885686 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.863004 2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 30 06:43:57.885686 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:57.863550 2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 30 06:43:57.885686 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:57.863595 2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-185.ec2.internal\" not found" Jul 30 06:43:57.908949 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.908874 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 30 06:43:57.910267 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.910251 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 30 06:43:57.910332 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.910288 2575 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 30 06:43:57.910332 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.910309 2575 kubelet.go:2448] "Starting kubelet main sync loop" Jul 30 06:43:57.910418 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:57.910342 2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 30 06:43:57.912537 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.912516 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 06:43:57.963496 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.963468 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 06:43:57.964362 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.964340 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientMemory" Jul 30 06:43:57.964474 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.964368 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 06:43:57.964474 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.964378 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientPID" Jul 30 06:43:57.964474 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.964402 2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-185.ec2.internal" Jul 30 06:43:57.970290 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:57.970275 2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-185.ec2.internal" Jul 30 06:43:57.970360 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:57.970296 2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-185.ec2.internal\": node \"ip-10-0-143-185.ec2.internal\" not found" Jul 30 06:43:57.994185 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:57.994152 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jul 30 06:43:58.010694 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.010669 2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal"] Jul 30 06:43:58.010787 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.010739 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 06:43:58.011805 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.011788 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientMemory" Jul 30 06:43:58.011887 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.011817 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 06:43:58.011887 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.011831 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientPID" Jul 30 06:43:58.014096 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.014082 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 06:43:58.014212 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.014196 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.014248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.014232 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 06:43:58.014882 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.014854 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientMemory" Jul 30 06:43:58.014967 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.014885 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 06:43:58.014967 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.014898 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientPID" Jul 30 06:43:58.014967 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.014858 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientMemory" Jul 30 06:43:58.014967 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.014961 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 06:43:58.015119 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.014973 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientPID" Jul 30 06:43:58.017106 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.017092 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.017161 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.017126 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 30 06:43:58.017835 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.017820 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientMemory" Jul 30 06:43:58.017898 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.017845 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasNoDiskPressure" Jul 30 06:43:58.017898 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.017859 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeHasSufficientPID" Jul 30 06:43:58.034372 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:58.034350 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-185.ec2.internal\" not found" node="ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.037947 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:58.037929 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-185.ec2.internal\" not found" node="ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.094851 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:58.094826 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jul 30 06:43:58.103306 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.103271 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/93b23d5a25ebd708496785d7e0e39b30-config\") pod \"kube-apiserver-proxy-ip-10-0-143-185.ec2.internal\" (UID: \"93b23d5a25ebd708496785d7e0e39b30\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.103371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.103315 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/88888b9b5c796691966ecbb4fc268aab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal\" (UID: \"88888b9b5c796691966ecbb4fc268aab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.103371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.103331 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/88888b9b5c796691966ecbb4fc268aab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal\" (UID: \"88888b9b5c796691966ecbb4fc268aab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.195571 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:58.195508 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jul 30 06:43:58.203925 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.203888 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/93b23d5a25ebd708496785d7e0e39b30-config\") pod \"kube-apiserver-proxy-ip-10-0-143-185.ec2.internal\" (UID: \"93b23d5a25ebd708496785d7e0e39b30\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.203925 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.203930 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/88888b9b5c796691966ecbb4fc268aab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal\" (UID: \"88888b9b5c796691966ecbb4fc268aab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.204024 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.203966 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/88888b9b5c796691966ecbb4fc268aab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal\" (UID: \"88888b9b5c796691966ecbb4fc268aab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.204024 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.204005 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/88888b9b5c796691966ecbb4fc268aab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal\" (UID: \"88888b9b5c796691966ecbb4fc268aab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.204024 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.204010 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/93b23d5a25ebd708496785d7e0e39b30-config\") pod \"kube-apiserver-proxy-ip-10-0-143-185.ec2.internal\" (UID: \"93b23d5a25ebd708496785d7e0e39b30\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.204114 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.204026 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/88888b9b5c796691966ecbb4fc268aab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal\" (UID: \"88888b9b5c796691966ecbb4fc268aab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.296379 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:58.296344 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jul 30 06:43:58.337958 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.337920 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.341856 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.341833 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.396895 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:58.396867 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jul 30 06:43:58.497424 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:58.497357 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jul 30 06:43:58.597993 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:58.597958 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jul 30 06:43:58.698480 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:58.698448 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jul 30 06:43:58.732037 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.732006 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 30 06:43:58.732581 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.732162 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 30 06:43:58.799132 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:43:58.799113 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-185.ec2.internal\" not found" Jul 30 06:43:58.821489 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.821436 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 06:38:59 +0000 UTC" deadline="2028-05-01 16:12:50.524287084 +0000 UTC" Jul 30 06:43:58.821489 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.821468 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15393h28m51.702821995s" Jul 30 06:43:58.884703 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.884676 2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 06:43:58.900169 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.900148 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.909390 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.909365 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 30 06:43:58.910962 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.910944 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" Jul 30 06:43:58.921731 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.921715 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 30 06:43:58.924919 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:58.924896 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93b23d5a25ebd708496785d7e0e39b30.slice/crio-d0a8b8cbc92d39b3828a31dc5924bbc13e31264ad2c88b078a43c51d90ea3788 WatchSource:0}: Error finding container d0a8b8cbc92d39b3828a31dc5924bbc13e31264ad2c88b078a43c51d90ea3788: Status 404 returned error can't find the container with id d0a8b8cbc92d39b3828a31dc5924bbc13e31264ad2c88b078a43c51d90ea3788 Jul 30 06:43:58.925377 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:43:58.925356 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88888b9b5c796691966ecbb4fc268aab.slice/crio-f93c5992ad9beca790692ee242b413660eece30b51f8214b828b6fe7079a01e0 WatchSource:0}: Error finding container f93c5992ad9beca790692ee242b413660eece30b51f8214b828b6fe7079a01e0: Status 404 returned error can't find the container with id f93c5992ad9beca790692ee242b413660eece30b51f8214b828b6fe7079a01e0 Jul 30 06:43:58.928882 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:58.928831 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 06:43:59.110849 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:43:59.110774 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 06:44:00.545784 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:00.545753 2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 06:44:01.035476 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.035205 2575 apiserver.go:52] "Watching apiserver" Jul 30 06:44:01.048578 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.048553 2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 30 06:44:01.048999 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.048977 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s","openshift-dns/node-resolver-7p8zb","openshift-multus/multus-additional-cni-plugins-v7s2f","openshift-ovn-kubernetes/ovnkube-node-84tq9","kube-system/global-pull-secret-syncer-sc2rl","openshift-cluster-node-tuning-operator/tuned-jqjxs","openshift-image-registry/node-ca-xmcq2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal","openshift-multus/multus-h785z","openshift-multus/network-metrics-daemon-7tcdf","openshift-network-diagnostics/network-check-target-fr6kg","openshift-network-operator/iptables-alerter-kkrfh","kube-system/konnectivity-agent-hdqsd","kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal"] Jul 30 06:44:01.052427 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.051789 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xmcq2" Jul 30 06:44:01.054191 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.054170 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hdqsd" Jul 30 06:44:01.055263 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.054860 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 30 06:44:01.055263 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.054889 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-86lxn\"" Jul 30 06:44:01.055263 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.055007 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 30 06:44:01.055263 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.055127 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 30 06:44:01.056224 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.056205 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.056490 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.056469 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 30 06:44:01.057047 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.057027 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 30 06:44:01.057235 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.057205 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rclc4\"" Jul 30 06:44:01.058524 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.058502 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 30 06:44:01.059476 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.058863 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fq54f\"" Jul 30 06:44:01.059476 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.059013 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 30 06:44:01.059476 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.059258 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 30 06:44:01.059476 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.059440 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 30 06:44:01.059712 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.059618 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 30 06:44:01.061342 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.059867 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.061342 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.060497 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 30 06:44:01.061843 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.061367 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:01.061843 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.061440 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:01.063255 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.063239 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.065992 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.065958 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 30 06:44:01.066280 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.066263 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 30 06:44:01.066347 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.066280 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 30 06:44:01.066479 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.066463 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 30 06:44:01.066519 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.066477 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 30 06:44:01.066638 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.066617 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h785z" Jul 30 06:44:01.066711 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.066694 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fpp9d\"" Jul 30 06:44:01.066879 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.066863 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 30 06:44:01.066997 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.066978 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 30 06:44:01.067112 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.067093 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 30 06:44:01.067172 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.067157 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rr6pd\"" Jul 30 06:44:01.068466 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.068446 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.069455 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.069438 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 30 06:44:01.069582 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.069569 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-78m9r\"" Jul 30 06:44:01.070743 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.070725 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 30 06:44:01.071030 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071013 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g594p\"" Jul 30 06:44:01.071251 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071233 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 30 06:44:01.071358 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071235 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 30 06:44:01.071562 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071545 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-var-lib-kubelet\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.071632 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071584 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htvhk\" (UniqueName: \"kubernetes.io/projected/7adaf671-8982-4075-92cd-d190635a2fe7-kube-api-access-htvhk\") pod \"node-ca-xmcq2\" (UID: \"7adaf671-8982-4075-92cd-d190635a2fe7\") " pod="openshift-image-registry/node-ca-xmcq2" Jul 30 06:44:01.071632 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071607 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7p8zb" Jul 30 06:44:01.071632 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071610 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7ce36c05-2aac-4717-8112-27dc9768ff02-agent-certs\") pod \"konnectivity-agent-hdqsd\" (UID: \"7ce36c05-2aac-4717-8112-27dc9768ff02\") " pod="kube-system/konnectivity-agent-hdqsd" Jul 30 06:44:01.071750 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071636 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c57d113e-75be-4b93-92f6-4c28e8a4714e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.071750 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071679 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-run-systemd\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.071750 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071704 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-var-lib-openvswitch\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.071750 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071736 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-run-ovn\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.071858 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071761 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-node-log\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.071858 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071788 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c57d113e-75be-4b93-92f6-4c28e8a4714e-os-release\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.071858 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071810 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-slash\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.071858 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071832 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-log-socket\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.071858 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071854 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-cni-bin\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.071997 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071877 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8e9d780d-93f9-40cd-a0f1-9577b8096849-ovnkube-config\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.071997 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071899 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8e9d780d-93f9-40cd-a0f1-9577b8096849-env-overrides\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.071997 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071921 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-dbus\") pod \"global-pull-secret-syncer-sc2rl\" (UID: \"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d\") " pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:01.071997 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071942 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7adaf671-8982-4075-92cd-d190635a2fe7-host\") pod \"node-ca-xmcq2\" (UID: \"7adaf671-8982-4075-92cd-d190635a2fe7\") " pod="openshift-image-registry/node-ca-xmcq2" Jul 30 06:44:01.071997 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071964 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7adaf671-8982-4075-92cd-d190635a2fe7-serviceca\") pod \"node-ca-xmcq2\" (UID: \"7adaf671-8982-4075-92cd-d190635a2fe7\") " pod="openshift-image-registry/node-ca-xmcq2" Jul 30 06:44:01.072152 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.071982 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c57d113e-75be-4b93-92f6-4c28e8a4714e-cnibin\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.072152 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.072024 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c57d113e-75be-4b93-92f6-4c28e8a4714e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.072152 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.072051 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-run-netns\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.072152 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.072073 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-run-openvswitch\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.072152 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.072097 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-sysctl-conf\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.072152 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.072120 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-tuned\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.072152 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.072144 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngmq9\" (UniqueName: \"kubernetes.io/projected/c57d113e-75be-4b93-92f6-4c28e8a4714e-kube-api-access-ngmq9\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.072405 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.072170 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-systemd-units\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.072405 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.072202 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k5d2\" (UniqueName: \"kubernetes.io/projected/8e9d780d-93f9-40cd-a0f1-9577b8096849-kube-api-access-8k5d2\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.072405 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.072224 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-sys\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.072405 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.072246 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7ceee03-f5e3-40fe-93a7-073ee217ae14-tmp\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.072405 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.072270 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c57d113e-75be-4b93-92f6-4c28e8a4714e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.072405 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.072294 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-etc-openvswitch\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.072405 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.072317 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e9d780d-93f9-40cd-a0f1-9577b8096849-ovn-node-metrics-cert\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.072405 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.072341 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8e9d780d-93f9-40cd-a0f1-9577b8096849-ovnkube-script-lib\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.072405 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.072386 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-kubelet-config\") pod \"global-pull-secret-syncer-sc2rl\" (UID: \"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d\") " pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:01.072908 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.072419 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret\") pod \"global-pull-secret-syncer-sc2rl\" (UID: \"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d\") " pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:01.073551 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.073530 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-lib-modules\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.073616 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.073566 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-host\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.073616 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.073602 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-cni-netd\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.073757 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.073630 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-systemd\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.073757 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.073669 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg2np\" (UniqueName: \"kubernetes.io/projected/c7ceee03-f5e3-40fe-93a7-073ee217ae14-kube-api-access-tg2np\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.073757 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.073694 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c57d113e-75be-4b93-92f6-4c28e8a4714e-system-cni-dir\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.073757 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.073718 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c57d113e-75be-4b93-92f6-4c28e8a4714e-cni-binary-copy\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.073757 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.073752 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-kubelet\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.074042 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.073780 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.074042 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.073812 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-modprobe-d\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.074042 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.073836 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-sysconfig\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.074042 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.073851 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-kubernetes\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.074042 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.073865 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-sysctl-d\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.074042 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.073891 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7ce36c05-2aac-4717-8112-27dc9768ff02-konnectivity-ca\") pod \"konnectivity-agent-hdqsd\" (UID: \"7ce36c05-2aac-4717-8112-27dc9768ff02\") " pod="kube-system/konnectivity-agent-hdqsd" Jul 30 06:44:01.074042 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.073916 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-run-ovn-kubernetes\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.074042 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.073939 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-run\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.074927 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.074908 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rx85m\"" Jul 30 06:44:01.075432 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.075413 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 30 06:44:01.075525 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.075507 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 30 06:44:01.075719 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.075703 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:01.075820 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.075787 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:01.076537 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.076511 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:01.076638 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.076575 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:01.077648 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.077626 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 30 06:44:01.078968 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.078937 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kkrfh" Jul 30 06:44:01.082183 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.082160 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 30 06:44:01.082542 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.082520 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 30 06:44:01.082765 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.082746 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 30 06:44:01.082836 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.082825 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2r9tq\"" Jul 30 06:44:01.099066 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.099042 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-n8kw5" Jul 30 06:44:01.105188 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.104866 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-n8kw5" Jul 30 06:44:01.155846 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.155817 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 30 06:44:01.161779 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.161753 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 30 06:44:01.174397 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174367 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-run-k8s-cni-cncf-io\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.174519 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174401 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-var-lib-kubelet\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.174519 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174421 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-run-multus-certs\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.174519 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174444 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9-hosts-file\") pod \"node-resolver-7p8zb\" (UID: \"2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9\") " pod="openshift-dns/node-resolver-7p8zb" Jul 30 06:44:01.174519 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174498 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9ct4\" (UniqueName: \"kubernetes.io/projected/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-kube-api-access-v9ct4\") pod \"network-metrics-daemon-7tcdf\" (UID: \"2a6fa524-6766-4ecd-a86a-5f26f29a72b2\") " pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:01.174734 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174549 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7adaf671-8982-4075-92cd-d190635a2fe7-host\") pod \"node-ca-xmcq2\" (UID: \"7adaf671-8982-4075-92cd-d190635a2fe7\") " pod="openshift-image-registry/node-ca-xmcq2" Jul 30 06:44:01.174734 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174568 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-tuned\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.174734 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174584 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txzlz\" (UniqueName: \"kubernetes.io/projected/961a0a68-4f71-40b4-926a-d9d3fda38987-kube-api-access-txzlz\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.174734 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174602 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-systemd-units\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.174734 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174624 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k5d2\" (UniqueName: \"kubernetes.io/projected/8e9d780d-93f9-40cd-a0f1-9577b8096849-kube-api-access-8k5d2\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.174734 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174646 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-sys\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.174734 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174688 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/03374672-1c6c-4996-ae88-a5fa11c92a57-iptables-alerter-script\") pod \"iptables-alerter-kkrfh\" (UID: \"03374672-1c6c-4996-ae88-a5fa11c92a57\") " pod="openshift-network-operator/iptables-alerter-kkrfh" Jul 30 06:44:01.174734 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174703 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-systemd-units\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.174734 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174707 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7adaf671-8982-4075-92cd-d190635a2fe7-host\") pod \"node-ca-xmcq2\" (UID: \"7adaf671-8982-4075-92cd-d190635a2fe7\") " pod="openshift-image-registry/node-ca-xmcq2" Jul 30 06:44:01.174734 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174714 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-kubelet-config\") pod \"global-pull-secret-syncer-sc2rl\" (UID: \"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d\") " pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174760 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-lib-modules\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174766 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-sys\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174774 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-kubelet-config\") pod \"global-pull-secret-syncer-sc2rl\" (UID: \"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d\") " pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174803 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg2np\" (UniqueName: \"kubernetes.io/projected/c7ceee03-f5e3-40fe-93a7-073ee217ae14-kube-api-access-tg2np\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174825 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-registration-dir\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174865 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-systemd\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174890 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-lib-modules\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174894 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-device-dir\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174907 2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174925 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-system-cni-dir\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174952 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/961a0a68-4f71-40b4-926a-d9d3fda38987-multus-daemon-config\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174959 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-systemd\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.174974 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9-tmp-dir\") pod \"node-resolver-7p8zb\" (UID: \"2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9\") " pod="openshift-dns/node-resolver-7p8zb" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175004 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-kubelet\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175031 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-modprobe-d\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175059 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-kubernetes\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175079 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-kubelet\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.175202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175084 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-var-lib-cni-bin\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175116 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-run-ovn-kubernetes\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175129 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-kubernetes\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175142 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/961a0a68-4f71-40b4-926a-d9d3fda38987-cni-binary-copy\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175174 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-multus-conf-dir\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175199 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm64h\" (UniqueName: \"kubernetes.io/projected/2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9-kube-api-access-nm64h\") pod \"node-resolver-7p8zb\" (UID: \"2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9\") " pod="openshift-dns/node-resolver-7p8zb" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175214 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-modprobe-d\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175218 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-run-ovn-kubernetes\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175230 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs\") pod \"network-metrics-daemon-7tcdf\" (UID: \"2a6fa524-6766-4ecd-a86a-5f26f29a72b2\") " pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175256 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-node-log\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175292 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c57d113e-75be-4b93-92f6-4c28e8a4714e-os-release\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175315 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-slash\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175354 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-cni-bin\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175360 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-node-log\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175378 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8e9d780d-93f9-40cd-a0f1-9577b8096849-env-overrides\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175392 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c57d113e-75be-4b93-92f6-4c28e8a4714e-os-release\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175402 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-slash\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.176041 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175402 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-dbus\") pod \"global-pull-secret-syncer-sc2rl\" (UID: \"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d\") " pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175438 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-sysctl-conf\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175460 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-etc-selinux\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175484 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lqsw\" (UniqueName: \"kubernetes.io/projected/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-kube-api-access-7lqsw\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175505 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7adaf671-8982-4075-92cd-d190635a2fe7-serviceca\") pod \"node-ca-xmcq2\" (UID: \"7adaf671-8982-4075-92cd-d190635a2fe7\") " pod="openshift-image-registry/node-ca-xmcq2" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175524 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c57d113e-75be-4b93-92f6-4c28e8a4714e-cnibin\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175543 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c57d113e-75be-4b93-92f6-4c28e8a4714e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175561 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-run-netns\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175581 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-run-openvswitch\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175599 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7ceee03-f5e3-40fe-93a7-073ee217ae14-tmp\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175617 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-cnibin\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175613 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-dbus\") pod \"global-pull-secret-syncer-sc2rl\" (UID: \"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d\") " pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175649 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-os-release\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175695 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-sysctl-conf\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175708 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngmq9\" (UniqueName: \"kubernetes.io/projected/c57d113e-75be-4b93-92f6-4c28e8a4714e-kube-api-access-ngmq9\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175736 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8src\" (UniqueName: \"kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src\") pod \"network-check-target-fr6kg\" (UID: \"b5281138-d6b6-4c65-a18f-6b153031868e\") " pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175763 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c57d113e-75be-4b93-92f6-4c28e8a4714e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.176851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175788 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-etc-openvswitch\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175816 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e9d780d-93f9-40cd-a0f1-9577b8096849-ovn-node-metrics-cert\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175840 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8e9d780d-93f9-40cd-a0f1-9577b8096849-ovnkube-script-lib\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175879 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8e9d780d-93f9-40cd-a0f1-9577b8096849-env-overrides\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.175980 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175989 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-etc-openvswitch\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176007 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7adaf671-8982-4075-92cd-d190635a2fe7-serviceca\") pod \"node-ca-xmcq2\" (UID: \"7adaf671-8982-4075-92cd-d190635a2fe7\") " pod="openshift-image-registry/node-ca-xmcq2" Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.176046 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret podName:dd5d594e-9943-4d46-97e0-44cbbdc6ff6d nodeName:}" failed. No retries permitted until 2026-07-30 06:44:01.676027338 +0000 UTC m=+3.033031491 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret") pod "global-pull-secret-syncer-sc2rl" (UID: "dd5d594e-9943-4d46-97e0-44cbbdc6ff6d") : object "kube-system"/"original-pull-secret" not registered Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176047 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c57d113e-75be-4b93-92f6-4c28e8a4714e-cnibin\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176094 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-cni-bin\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.175882 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret\") pod \"global-pull-secret-syncer-sc2rl\" (UID: \"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d\") " pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176129 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-host\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176155 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-run-netns\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176166 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c57d113e-75be-4b93-92f6-4c28e8a4714e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176177 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-hostroot\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176202 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-cni-netd\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176211 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-run-netns\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.177621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176228 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-socket-dir\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176254 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03374672-1c6c-4996-ae88-a5fa11c92a57-host-slash\") pod \"iptables-alerter-kkrfh\" (UID: \"03374672-1c6c-4996-ae88-a5fa11c92a57\") " pod="openshift-network-operator/iptables-alerter-kkrfh" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176278 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-multus-cni-dir\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176305 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c57d113e-75be-4b93-92f6-4c28e8a4714e-system-cni-dir\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176331 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c57d113e-75be-4b93-92f6-4c28e8a4714e-cni-binary-copy\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176360 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176387 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-sysconfig\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176411 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-sysctl-d\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176437 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-var-lib-cni-multus\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176457 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c57d113e-75be-4b93-92f6-4c28e8a4714e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176497 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8e9d780d-93f9-40cd-a0f1-9577b8096849-ovnkube-script-lib\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176520 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-host\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176463 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7ce36c05-2aac-4717-8112-27dc9768ff02-konnectivity-ca\") pod \"konnectivity-agent-hdqsd\" (UID: \"7ce36c05-2aac-4717-8112-27dc9768ff02\") " pod="kube-system/konnectivity-agent-hdqsd" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176553 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-run\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176577 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-var-lib-kubelet\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176605 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8njm\" (UniqueName: \"kubernetes.io/projected/03374672-1c6c-4996-ae88-a5fa11c92a57-kube-api-access-z8njm\") pod \"iptables-alerter-kkrfh\" (UID: \"03374672-1c6c-4996-ae88-a5fa11c92a57\") " pod="openshift-network-operator/iptables-alerter-kkrfh" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176629 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-multus-socket-dir-parent\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.178506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176679 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-etc-kubernetes\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176707 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htvhk\" (UniqueName: \"kubernetes.io/projected/7adaf671-8982-4075-92cd-d190635a2fe7-kube-api-access-htvhk\") pod \"node-ca-xmcq2\" (UID: \"7adaf671-8982-4075-92cd-d190635a2fe7\") " pod="openshift-image-registry/node-ca-xmcq2" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176733 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7ce36c05-2aac-4717-8112-27dc9768ff02-agent-certs\") pod \"konnectivity-agent-hdqsd\" (UID: \"7ce36c05-2aac-4717-8112-27dc9768ff02\") " pod="kube-system/konnectivity-agent-hdqsd" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176761 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c57d113e-75be-4b93-92f6-4c28e8a4714e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176787 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-run-systemd\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176833 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-var-lib-openvswitch\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176859 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-run-ovn\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176917 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176950 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-log-socket\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176954 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-sysconfig\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176978 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176413 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-run-openvswitch\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.177015 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8e9d780d-93f9-40cd-a0f1-9577b8096849-ovnkube-config\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.177027 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-run\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.177039 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-sys-fs\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.177085 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-sysctl-d\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.177165 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-host-cni-netd\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.189090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.177177 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-run-ovn\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.177224 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-log-socket\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.176978 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c57d113e-75be-4b93-92f6-4c28e8a4714e-cni-binary-copy\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.177501 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7ce36c05-2aac-4717-8112-27dc9768ff02-konnectivity-ca\") pod \"konnectivity-agent-hdqsd\" (UID: \"7ce36c05-2aac-4717-8112-27dc9768ff02\") " pod="kube-system/konnectivity-agent-hdqsd" Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.177539 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c57d113e-75be-4b93-92f6-4c28e8a4714e-system-cni-dir\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.177576 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7ceee03-f5e3-40fe-93a7-073ee217ae14-var-lib-kubelet\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.177625 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-run-systemd\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.177694 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e9d780d-93f9-40cd-a0f1-9577b8096849-var-lib-openvswitch\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.177864 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8e9d780d-93f9-40cd-a0f1-9577b8096849-ovnkube-config\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.178163 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c57d113e-75be-4b93-92f6-4c28e8a4714e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.179960 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c7ceee03-f5e3-40fe-93a7-073ee217ae14-etc-tuned\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.180015 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7ceee03-f5e3-40fe-93a7-073ee217ae14-tmp\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.180805 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" event={"ID":"88888b9b5c796691966ecbb4fc268aab","Type":"ContainerStarted","Data":"f93c5992ad9beca790692ee242b413660eece30b51f8214b828b6fe7079a01e0"} Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.181531 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7ce36c05-2aac-4717-8112-27dc9768ff02-agent-certs\") pod \"konnectivity-agent-hdqsd\" (UID: \"7ce36c05-2aac-4717-8112-27dc9768ff02\") " pod="kube-system/konnectivity-agent-hdqsd" Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.182070 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" event={"ID":"93b23d5a25ebd708496785d7e0e39b30","Type":"ContainerStarted","Data":"d0a8b8cbc92d39b3828a31dc5924bbc13e31264ad2c88b078a43c51d90ea3788"} Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.182773 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e9d780d-93f9-40cd-a0f1-9577b8096849-ovn-node-metrics-cert\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.185876 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg2np\" (UniqueName: \"kubernetes.io/projected/c7ceee03-f5e3-40fe-93a7-073ee217ae14-kube-api-access-tg2np\") pod \"tuned-jqjxs\" (UID: \"c7ceee03-f5e3-40fe-93a7-073ee217ae14\") " pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.187601 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htvhk\" (UniqueName: \"kubernetes.io/projected/7adaf671-8982-4075-92cd-d190635a2fe7-kube-api-access-htvhk\") pod \"node-ca-xmcq2\" (UID: \"7adaf671-8982-4075-92cd-d190635a2fe7\") " pod="openshift-image-registry/node-ca-xmcq2" Jul 30 06:44:01.189960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.188236 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k5d2\" (UniqueName: \"kubernetes.io/projected/8e9d780d-93f9-40cd-a0f1-9577b8096849-kube-api-access-8k5d2\") pod \"ovnkube-node-84tq9\" (UID: \"8e9d780d-93f9-40cd-a0f1-9577b8096849\") " pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.190835 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.189052 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngmq9\" (UniqueName: \"kubernetes.io/projected/c57d113e-75be-4b93-92f6-4c28e8a4714e-kube-api-access-ngmq9\") pod \"multus-additional-cni-plugins-v7s2f\" (UID: \"c57d113e-75be-4b93-92f6-4c28e8a4714e\") " pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.277534 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277490 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-etc-selinux\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.277534 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277534 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lqsw\" (UniqueName: \"kubernetes.io/projected/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-kube-api-access-7lqsw\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.277860 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277565 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-cnibin\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.277860 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277586 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-os-release\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.277860 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277608 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8src\" (UniqueName: \"kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src\") pod \"network-check-target-fr6kg\" (UID: \"b5281138-d6b6-4c65-a18f-6b153031868e\") " pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:01.277860 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277649 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-run-netns\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.277860 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277691 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-hostroot\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.277860 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277713 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-socket-dir\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.277860 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277764 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03374672-1c6c-4996-ae88-a5fa11c92a57-host-slash\") pod \"iptables-alerter-kkrfh\" (UID: \"03374672-1c6c-4996-ae88-a5fa11c92a57\") " pod="openshift-network-operator/iptables-alerter-kkrfh" Jul 30 06:44:01.277860 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277790 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-multus-cni-dir\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.277860 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277828 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-var-lib-cni-multus\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.277860 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277852 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-os-release\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277855 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8njm\" (UniqueName: \"kubernetes.io/projected/03374672-1c6c-4996-ae88-a5fa11c92a57-kube-api-access-z8njm\") pod \"iptables-alerter-kkrfh\" (UID: \"03374672-1c6c-4996-ae88-a5fa11c92a57\") " pod="openshift-network-operator/iptables-alerter-kkrfh" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277911 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-multus-socket-dir-parent\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277937 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-etc-kubernetes\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277970 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.277997 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-sys-fs\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278020 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-run-k8s-cni-cncf-io\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278045 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-var-lib-kubelet\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278068 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-run-multus-certs\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278091 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9-hosts-file\") pod \"node-resolver-7p8zb\" (UID: \"2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9\") " pod="openshift-dns/node-resolver-7p8zb" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278114 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9ct4\" (UniqueName: \"kubernetes.io/projected/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-kube-api-access-v9ct4\") pod \"network-metrics-daemon-7tcdf\" (UID: \"2a6fa524-6766-4ecd-a86a-5f26f29a72b2\") " pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278135 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txzlz\" (UniqueName: \"kubernetes.io/projected/961a0a68-4f71-40b4-926a-d9d3fda38987-kube-api-access-txzlz\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278152 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/03374672-1c6c-4996-ae88-a5fa11c92a57-iptables-alerter-script\") pod \"iptables-alerter-kkrfh\" (UID: \"03374672-1c6c-4996-ae88-a5fa11c92a57\") " pod="openshift-network-operator/iptables-alerter-kkrfh" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278175 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-registration-dir\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278197 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-device-dir\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278211 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-system-cni-dir\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278225 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/961a0a68-4f71-40b4-926a-d9d3fda38987-multus-daemon-config\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.278371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278246 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9-tmp-dir\") pod \"node-resolver-7p8zb\" (UID: \"2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9\") " pod="openshift-dns/node-resolver-7p8zb" Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278268 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-var-lib-cni-bin\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278288 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/961a0a68-4f71-40b4-926a-d9d3fda38987-cni-binary-copy\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278298 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-hostroot\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278310 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-multus-conf-dir\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278336 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm64h\" (UniqueName: \"kubernetes.io/projected/2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9-kube-api-access-nm64h\") pod \"node-resolver-7p8zb\" (UID: \"2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9\") " pod="openshift-dns/node-resolver-7p8zb" Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278360 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs\") pod \"network-metrics-daemon-7tcdf\" (UID: \"2a6fa524-6766-4ecd-a86a-5f26f29a72b2\") " pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278443 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-socket-dir\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.278475 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278496 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03374672-1c6c-4996-ae88-a5fa11c92a57-host-slash\") pod \"iptables-alerter-kkrfh\" (UID: \"03374672-1c6c-4996-ae88-a5fa11c92a57\") " pod="openshift-network-operator/iptables-alerter-kkrfh" Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.278529 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs podName:2a6fa524-6766-4ecd-a86a-5f26f29a72b2 nodeName:}" failed. No retries permitted until 2026-07-30 06:44:01.778512032 +0000 UTC m=+3.135516175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs") pod "network-metrics-daemon-7tcdf" (UID: "2a6fa524-6766-4ecd-a86a-5f26f29a72b2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278557 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-multus-cni-dir\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278600 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-var-lib-cni-multus\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.278898 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-etc-selinux\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279122 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-cnibin\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279175 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-multus-socket-dir-parent\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279218 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-etc-kubernetes\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.279248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279238 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/03374672-1c6c-4996-ae88-a5fa11c92a57-iptables-alerter-script\") pod \"iptables-alerter-kkrfh\" (UID: \"03374672-1c6c-4996-ae88-a5fa11c92a57\") " pod="openshift-network-operator/iptables-alerter-kkrfh" Jul 30 06:44:01.280119 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279261 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.280119 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279316 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-registration-dir\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.280119 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279318 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-sys-fs\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.280119 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279362 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-device-dir\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.280119 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279362 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-run-k8s-cni-cncf-io\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.280119 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279397 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-var-lib-kubelet\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.280119 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279414 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-system-cni-dir\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.280119 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279447 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-run-multus-certs\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.280119 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279548 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9-hosts-file\") pod \"node-resolver-7p8zb\" (UID: \"2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9\") " pod="openshift-dns/node-resolver-7p8zb" Jul 30 06:44:01.280119 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279588 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-var-lib-cni-bin\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.280119 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279636 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-host-run-netns\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.280119 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279640 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/961a0a68-4f71-40b4-926a-d9d3fda38987-multus-conf-dir\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.280119 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.279812 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9-tmp-dir\") pod \"node-resolver-7p8zb\" (UID: \"2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9\") " pod="openshift-dns/node-resolver-7p8zb" Jul 30 06:44:01.280765 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.280384 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/961a0a68-4f71-40b4-926a-d9d3fda38987-multus-daemon-config\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.280765 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.280469 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/961a0a68-4f71-40b4-926a-d9d3fda38987-cni-binary-copy\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.283694 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.283672 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 06:44:01.283781 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.283699 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 06:44:01.283781 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.283714 2575 projected.go:196] Error preparing data for projected volume kube-api-access-l8src for pod openshift-network-diagnostics/network-check-target-fr6kg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 06:44:01.283781 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.283777 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src podName:b5281138-d6b6-4c65-a18f-6b153031868e nodeName:}" failed. No retries permitted until 2026-07-30 06:44:01.783762412 +0000 UTC m=+3.140766557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-l8src" (UniqueName: "kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src") pod "network-check-target-fr6kg" (UID: "b5281138-d6b6-4c65-a18f-6b153031868e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 06:44:01.286903 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.286855 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8njm\" (UniqueName: \"kubernetes.io/projected/03374672-1c6c-4996-ae88-a5fa11c92a57-kube-api-access-z8njm\") pod \"iptables-alerter-kkrfh\" (UID: \"03374672-1c6c-4996-ae88-a5fa11c92a57\") " pod="openshift-network-operator/iptables-alerter-kkrfh" Jul 30 06:44:01.287124 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.287081 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txzlz\" (UniqueName: \"kubernetes.io/projected/961a0a68-4f71-40b4-926a-d9d3fda38987-kube-api-access-txzlz\") pod \"multus-h785z\" (UID: \"961a0a68-4f71-40b4-926a-d9d3fda38987\") " pod="openshift-multus/multus-h785z" Jul 30 06:44:01.288513 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.288492 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9ct4\" (UniqueName: \"kubernetes.io/projected/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-kube-api-access-v9ct4\") pod \"network-metrics-daemon-7tcdf\" (UID: \"2a6fa524-6766-4ecd-a86a-5f26f29a72b2\") " pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:01.288972 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.288948 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lqsw\" (UniqueName: \"kubernetes.io/projected/a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a-kube-api-access-7lqsw\") pod \"aws-ebs-csi-driver-node-vkj6s\" (UID: \"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.289432 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.289409 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm64h\" (UniqueName: \"kubernetes.io/projected/2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9-kube-api-access-nm64h\") pod \"node-resolver-7p8zb\" (UID: \"2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9\") " pod="openshift-dns/node-resolver-7p8zb" Jul 30 06:44:01.366561 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.366524 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xmcq2" Jul 30 06:44:01.377672 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.377625 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hdqsd" Jul 30 06:44:01.393774 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.393744 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v7s2f" Jul 30 06:44:01.398523 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.398496 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kkrfh" Jul 30 06:44:01.406119 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.406100 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:01.411957 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.411939 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" Jul 30 06:44:01.420711 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.420690 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h785z" Jul 30 06:44:01.427636 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.427616 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" Jul 30 06:44:01.433358 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.433337 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7p8zb" Jul 30 06:44:01.680713 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.680628 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret\") pod \"global-pull-secret-syncer-sc2rl\" (UID: \"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d\") " pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:01.680849 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.680775 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 06:44:01.680892 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.680853 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret podName:dd5d594e-9943-4d46-97e0-44cbbdc6ff6d nodeName:}" failed. No retries permitted until 2026-07-30 06:44:02.680830139 +0000 UTC m=+4.037834285 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret") pod "global-pull-secret-syncer-sc2rl" (UID: "dd5d594e-9943-4d46-97e0-44cbbdc6ff6d") : object "kube-system"/"original-pull-secret" not registered Jul 30 06:44:01.781120 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.781089 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs\") pod \"network-metrics-daemon-7tcdf\" (UID: \"2a6fa524-6766-4ecd-a86a-5f26f29a72b2\") " pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:01.781300 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.781273 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 06:44:01.781381 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.781369 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs podName:2a6fa524-6766-4ecd-a86a-5f26f29a72b2 nodeName:}" failed. No retries permitted until 2026-07-30 06:44:02.781347195 +0000 UTC m=+4.138351342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs") pod "network-metrics-daemon-7tcdf" (UID: "2a6fa524-6766-4ecd-a86a-5f26f29a72b2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 06:44:01.882507 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:01.882471 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8src\" (UniqueName: \"kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src\") pod \"network-check-target-fr6kg\" (UID: \"b5281138-d6b6-4c65-a18f-6b153031868e\") " pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:01.882709 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.882635 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 06:44:01.882709 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.882667 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 06:44:01.882709 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.882684 2575 projected.go:196] Error preparing data for projected volume kube-api-access-l8src for pod openshift-network-diagnostics/network-check-target-fr6kg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 06:44:01.882877 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:01.882736 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src podName:b5281138-d6b6-4c65-a18f-6b153031868e nodeName:}" failed. No retries permitted until 2026-07-30 06:44:02.882723644 +0000 UTC m=+4.239727783 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-l8src" (UniqueName: "kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src") pod "network-check-target-fr6kg" (UID: "b5281138-d6b6-4c65-a18f-6b153031868e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 06:44:02.089927 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:44:02.089787 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03374672_1c6c_4996_ae88_a5fa11c92a57.slice/crio-f02596f247345029b999e24d1d930433ced617709ea09c2c2f360bbf764c1673 WatchSource:0}: Error finding container f02596f247345029b999e24d1d930433ced617709ea09c2c2f360bbf764c1673: Status 404 returned error can't find the container with id f02596f247345029b999e24d1d930433ced617709ea09c2c2f360bbf764c1673 Jul 30 06:44:02.091873 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:44:02.091611 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce36c05_2aac_4717_8112_27dc9768ff02.slice/crio-983b8067c0c7c3f6fdb496d4d7c0c6c7813b2f0dc8f2116bc8465ade2cb4d19b WatchSource:0}: Error finding container 983b8067c0c7c3f6fdb496d4d7c0c6c7813b2f0dc8f2116bc8465ade2cb4d19b: Status 404 returned error can't find the container with id 983b8067c0c7c3f6fdb496d4d7c0c6c7813b2f0dc8f2116bc8465ade2cb4d19b Jul 30 06:44:02.095329 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:44:02.094916 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e9d780d_93f9_40cd_a0f1_9577b8096849.slice/crio-a41781a2065218267bc1d73625e9e219058580db10cafe10c67901a49a35c578 WatchSource:0}: Error finding container a41781a2065218267bc1d73625e9e219058580db10cafe10c67901a49a35c578: Status 404 returned error can't find the container with id a41781a2065218267bc1d73625e9e219058580db10cafe10c67901a49a35c578 Jul 30 06:44:02.096745 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:44:02.096719 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6b4d2f1_d7eb_4577_be6e_6b1bd598ff2a.slice/crio-ba98394b68c89908ffc85aebc10b0abc49494999e32d8414f74ecbf8d598ab77 WatchSource:0}: Error finding container ba98394b68c89908ffc85aebc10b0abc49494999e32d8414f74ecbf8d598ab77: Status 404 returned error can't find the container with id ba98394b68c89908ffc85aebc10b0abc49494999e32d8414f74ecbf8d598ab77 Jul 30 06:44:02.097544 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:44:02.097527 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ba7155b_6db1_4d5e_aa46_a8cc4a2861f9.slice/crio-5689d9161e623209830a543e86f54772d6b438bd5655105a363b3121cc677bb2 WatchSource:0}: Error finding container 5689d9161e623209830a543e86f54772d6b438bd5655105a363b3121cc677bb2: Status 404 returned error can't find the container with id 5689d9161e623209830a543e86f54772d6b438bd5655105a363b3121cc677bb2 Jul 30 06:44:02.098251 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:44:02.098227 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7ceee03_f5e3_40fe_93a7_073ee217ae14.slice/crio-d6d00a4ebadc37005fe1614f681c287be2f33216f95a4645ec61817b8151c399 WatchSource:0}: Error finding container d6d00a4ebadc37005fe1614f681c287be2f33216f95a4645ec61817b8151c399: Status 404 returned error can't find the container with id d6d00a4ebadc37005fe1614f681c287be2f33216f95a4645ec61817b8151c399 Jul 30 06:44:02.099207 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:44:02.099188 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7adaf671_8982_4075_92cd_d190635a2fe7.slice/crio-b85f166d315692818586268549bd685ded6d9d99eccb1e87c2488601c2109e80 WatchSource:0}: Error finding container b85f166d315692818586268549bd685ded6d9d99eccb1e87c2488601c2109e80: Status 404 returned error can't find the container with id b85f166d315692818586268549bd685ded6d9d99eccb1e87c2488601c2109e80 Jul 30 06:44:02.108272 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:02.108244 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 06:39:01 +0000 UTC" deadline="2028-01-13 02:23:10.205272177 +0000 UTC" Jul 30 06:44:02.108272 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:02.108272 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12763h39m8.09700392s" Jul 30 06:44:02.120277 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:44:02.120249 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc57d113e_75be_4b93_92f6_4c28e8a4714e.slice/crio-daeafc2406b3494cd6b9e576432c50673d38615bfd852f72d3e258c04d2d334d WatchSource:0}: Error finding container daeafc2406b3494cd6b9e576432c50673d38615bfd852f72d3e258c04d2d334d: Status 404 returned error can't find the container with id daeafc2406b3494cd6b9e576432c50673d38615bfd852f72d3e258c04d2d334d Jul 30 06:44:02.121120 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:44:02.121095 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod961a0a68_4f71_40b4_926a_d9d3fda38987.slice/crio-869a4b03a7c9177bb3865511857f870884d725dd5ce42f5ae0365e3b3f03f12f WatchSource:0}: Error finding container 869a4b03a7c9177bb3865511857f870884d725dd5ce42f5ae0365e3b3f03f12f: Status 404 returned error can't find the container with id 869a4b03a7c9177bb3865511857f870884d725dd5ce42f5ae0365e3b3f03f12f Jul 30 06:44:02.170918 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:02.170788 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:02.171011 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:02.170848 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:02.171011 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:02.170995 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:02.171131 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:02.171095 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:02.189791 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:02.189765 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7p8zb" event={"ID":"2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9","Type":"ContainerStarted","Data":"5689d9161e623209830a543e86f54772d6b438bd5655105a363b3121cc677bb2"} Jul 30 06:44:02.191007 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:02.190974 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v7s2f" event={"ID":"c57d113e-75be-4b93-92f6-4c28e8a4714e","Type":"ContainerStarted","Data":"daeafc2406b3494cd6b9e576432c50673d38615bfd852f72d3e258c04d2d334d"} Jul 30 06:44:02.192109 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:02.192082 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" event={"ID":"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a","Type":"ContainerStarted","Data":"ba98394b68c89908ffc85aebc10b0abc49494999e32d8414f74ecbf8d598ab77"} Jul 30 06:44:02.193443 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:02.193420 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" event={"ID":"8e9d780d-93f9-40cd-a0f1-9577b8096849","Type":"ContainerStarted","Data":"a41781a2065218267bc1d73625e9e219058580db10cafe10c67901a49a35c578"} Jul 30 06:44:02.194561 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:02.194534 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xmcq2" event={"ID":"7adaf671-8982-4075-92cd-d190635a2fe7","Type":"ContainerStarted","Data":"b85f166d315692818586268549bd685ded6d9d99eccb1e87c2488601c2109e80"} Jul 30 06:44:02.195525 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:02.195500 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" event={"ID":"c7ceee03-f5e3-40fe-93a7-073ee217ae14","Type":"ContainerStarted","Data":"d6d00a4ebadc37005fe1614f681c287be2f33216f95a4645ec61817b8151c399"} Jul 30 06:44:02.196395 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:02.196365 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hdqsd" event={"ID":"7ce36c05-2aac-4717-8112-27dc9768ff02","Type":"ContainerStarted","Data":"983b8067c0c7c3f6fdb496d4d7c0c6c7813b2f0dc8f2116bc8465ade2cb4d19b"} Jul 30 06:44:02.197434 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:02.197408 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kkrfh" event={"ID":"03374672-1c6c-4996-ae88-a5fa11c92a57","Type":"ContainerStarted","Data":"f02596f247345029b999e24d1d930433ced617709ea09c2c2f360bbf764c1673"} Jul 30 06:44:02.198506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:02.198473 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h785z" event={"ID":"961a0a68-4f71-40b4-926a-d9d3fda38987","Type":"ContainerStarted","Data":"869a4b03a7c9177bb3865511857f870884d725dd5ce42f5ae0365e3b3f03f12f"} Jul 30 06:44:02.689951 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:02.689914 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret\") pod \"global-pull-secret-syncer-sc2rl\" (UID: \"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d\") " pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:02.690125 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:02.690099 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 06:44:02.690265 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:02.690191 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret podName:dd5d594e-9943-4d46-97e0-44cbbdc6ff6d nodeName:}" failed. No retries permitted until 2026-07-30 06:44:04.690164616 +0000 UTC m=+6.047168773 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret") pod "global-pull-secret-syncer-sc2rl" (UID: "dd5d594e-9943-4d46-97e0-44cbbdc6ff6d") : object "kube-system"/"original-pull-secret" not registered Jul 30 06:44:02.790849 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:02.790811 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs\") pod \"network-metrics-daemon-7tcdf\" (UID: \"2a6fa524-6766-4ecd-a86a-5f26f29a72b2\") " pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:02.791026 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:02.790993 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 06:44:02.791066 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:02.791056 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs podName:2a6fa524-6766-4ecd-a86a-5f26f29a72b2 nodeName:}" failed. No retries permitted until 2026-07-30 06:44:04.791037637 +0000 UTC m=+6.148041781 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs") pod "network-metrics-daemon-7tcdf" (UID: "2a6fa524-6766-4ecd-a86a-5f26f29a72b2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 06:44:02.892471 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:02.891866 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8src\" (UniqueName: \"kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src\") pod \"network-check-target-fr6kg\" (UID: \"b5281138-d6b6-4c65-a18f-6b153031868e\") " pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:02.892471 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:02.892050 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 06:44:02.892471 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:02.892070 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 06:44:02.892471 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:02.892083 2575 projected.go:196] Error preparing data for projected volume kube-api-access-l8src for pod openshift-network-diagnostics/network-check-target-fr6kg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 06:44:02.892471 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:02.892141 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src podName:b5281138-d6b6-4c65-a18f-6b153031868e nodeName:}" failed. No retries permitted until 2026-07-30 06:44:04.892123503 +0000 UTC m=+6.249127663 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-l8src" (UniqueName: "kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src") pod "network-check-target-fr6kg" (UID: "b5281138-d6b6-4c65-a18f-6b153031868e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 06:44:03.109188 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:03.109072 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 06:39:01 +0000 UTC" deadline="2028-05-15 22:00:55.127809144 +0000 UTC" Jul 30 06:44:03.109188 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:03.109135 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15735h16m52.018677615s" Jul 30 06:44:03.173530 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:03.172979 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:03.173530 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:03.173133 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:03.219766 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:03.219731 2575 generic.go:356] "Generic (PLEG): container finished" podID="88888b9b5c796691966ecbb4fc268aab" containerID="cc37edad9862ea9d16f3d72c51882b64fb0dc4527d0df52c4b8e3ba6644c289f" exitCode=0 Jul 30 06:44:03.219990 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:03.219915 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" event={"ID":"88888b9b5c796691966ecbb4fc268aab","Type":"ContainerDied","Data":"cc37edad9862ea9d16f3d72c51882b64fb0dc4527d0df52c4b8e3ba6644c289f"} Jul 30 06:44:03.233295 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:03.233110 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" event={"ID":"93b23d5a25ebd708496785d7e0e39b30","Type":"ContainerStarted","Data":"f60e0e56e57c724bff7d09f02b2af08218affa5f89db1b2fd92bb201a3204ced"} Jul 30 06:44:04.170955 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:04.170920 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:04.171437 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:04.171054 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:04.171519 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:04.171485 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:04.171611 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:04.171588 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:04.252982 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:04.251564 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" event={"ID":"88888b9b5c796691966ecbb4fc268aab","Type":"ContainerStarted","Data":"1ab9fbd093d754190ba5de677ce21238476dbdcf8887dcfdf0049bb50b858f47"} Jul 30 06:44:04.267413 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:04.266350 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-185.ec2.internal" podStartSLOduration=4.266329076 podStartE2EDuration="4.266329076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 06:44:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:44:03.252119551 +0000 UTC m=+4.609123715" watchObservedRunningTime="2026-07-30 06:44:04.266329076 +0000 UTC m=+5.623333241" Jul 30 06:44:04.707002 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:04.706963 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret\") pod \"global-pull-secret-syncer-sc2rl\" (UID: \"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d\") " pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:04.707199 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:04.707131 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 06:44:04.707199 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:04.707194 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret podName:dd5d594e-9943-4d46-97e0-44cbbdc6ff6d nodeName:}" failed. No retries permitted until 2026-07-30 06:44:08.707176475 +0000 UTC m=+10.064180626 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret") pod "global-pull-secret-syncer-sc2rl" (UID: "dd5d594e-9943-4d46-97e0-44cbbdc6ff6d") : object "kube-system"/"original-pull-secret" not registered Jul 30 06:44:04.808319 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:04.808279 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs\") pod \"network-metrics-daemon-7tcdf\" (UID: \"2a6fa524-6766-4ecd-a86a-5f26f29a72b2\") " pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:04.808508 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:04.808420 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 06:44:04.808508 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:04.808494 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs podName:2a6fa524-6766-4ecd-a86a-5f26f29a72b2 nodeName:}" failed. No retries permitted until 2026-07-30 06:44:08.808472945 +0000 UTC m=+10.165477107 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs") pod "network-metrics-daemon-7tcdf" (UID: "2a6fa524-6766-4ecd-a86a-5f26f29a72b2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 06:44:04.909856 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:04.909257 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8src\" (UniqueName: \"kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src\") pod \"network-check-target-fr6kg\" (UID: \"b5281138-d6b6-4c65-a18f-6b153031868e\") " pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:04.909856 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:04.909461 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 06:44:04.909856 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:04.909482 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 06:44:04.909856 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:04.909495 2575 projected.go:196] Error preparing data for projected volume kube-api-access-l8src for pod openshift-network-diagnostics/network-check-target-fr6kg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 06:44:04.909856 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:04.909550 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src podName:b5281138-d6b6-4c65-a18f-6b153031868e nodeName:}" failed. No retries permitted until 2026-07-30 06:44:08.909534201 +0000 UTC m=+10.266538344 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-l8src" (UniqueName: "kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src") pod "network-check-target-fr6kg" (UID: "b5281138-d6b6-4c65-a18f-6b153031868e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 06:44:05.173541 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:05.172801 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:05.173541 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:05.172949 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:06.171051 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:06.171015 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:06.171254 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:06.171146 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:06.171572 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:06.171526 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:06.171681 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:06.171616 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:07.173205 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:07.171843 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:07.173205 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:07.171970 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:08.170726 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:08.170693 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:08.170905 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:08.170697 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:08.170905 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:08.170877 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:08.171022 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:08.170945 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:08.743406 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:08.743365 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret\") pod \"global-pull-secret-syncer-sc2rl\" (UID: \"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d\") " pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:08.743888 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:08.743545 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 06:44:08.743888 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:08.743610 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret podName:dd5d594e-9943-4d46-97e0-44cbbdc6ff6d nodeName:}" failed. No retries permitted until 2026-07-30 06:44:16.743592856 +0000 UTC m=+18.100596999 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret") pod "global-pull-secret-syncer-sc2rl" (UID: "dd5d594e-9943-4d46-97e0-44cbbdc6ff6d") : object "kube-system"/"original-pull-secret" not registered Jul 30 06:44:08.844669 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:08.844580 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs\") pod \"network-metrics-daemon-7tcdf\" (UID: \"2a6fa524-6766-4ecd-a86a-5f26f29a72b2\") " pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:08.844839 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:08.844784 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 06:44:08.844900 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:08.844863 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs podName:2a6fa524-6766-4ecd-a86a-5f26f29a72b2 nodeName:}" failed. No retries permitted until 2026-07-30 06:44:16.844846751 +0000 UTC m=+18.201850895 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs") pod "network-metrics-daemon-7tcdf" (UID: "2a6fa524-6766-4ecd-a86a-5f26f29a72b2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 06:44:08.945672 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:08.945612 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8src\" (UniqueName: \"kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src\") pod \"network-check-target-fr6kg\" (UID: \"b5281138-d6b6-4c65-a18f-6b153031868e\") " pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:08.946261 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:08.945835 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 06:44:08.946261 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:08.945859 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 06:44:08.946261 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:08.945871 2575 projected.go:196] Error preparing data for projected volume kube-api-access-l8src for pod openshift-network-diagnostics/network-check-target-fr6kg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 06:44:08.946261 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:08.945931 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src podName:b5281138-d6b6-4c65-a18f-6b153031868e nodeName:}" failed. No retries permitted until 2026-07-30 06:44:16.945912515 +0000 UTC m=+18.302916678 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-l8src" (UniqueName: "kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src") pod "network-check-target-fr6kg" (UID: "b5281138-d6b6-4c65-a18f-6b153031868e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 06:44:09.175880 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:09.175853 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:09.176039 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:09.175996 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:10.171203 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:10.171168 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:10.171643 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:10.171323 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:10.171745 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:10.171708 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:10.171853 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:10.171825 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:11.173483 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:11.173448 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:11.173909 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:11.173600 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:12.171454 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:12.171410 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:12.171454 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:12.171452 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:12.171724 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:12.171559 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:12.171724 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:12.171644 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:13.171130 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:13.171098 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:13.171507 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:13.171249 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:14.171011 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:14.170978 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:14.171162 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:14.171025 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:14.171162 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:14.171117 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:14.171505 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:14.171245 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:15.171294 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:15.171257 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:15.171718 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:15.171387 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:16.170716 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:16.170677 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:16.170716 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:16.170722 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:16.170921 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:16.170806 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:16.170960 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:16.170927 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:16.806175 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:16.806128 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret\") pod \"global-pull-secret-syncer-sc2rl\" (UID: \"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d\") " pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:16.806616 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:16.806263 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 06:44:16.806616 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:16.806326 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret podName:dd5d594e-9943-4d46-97e0-44cbbdc6ff6d nodeName:}" failed. No retries permitted until 2026-07-30 06:44:32.806306798 +0000 UTC m=+34.163310938 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret") pod "global-pull-secret-syncer-sc2rl" (UID: "dd5d594e-9943-4d46-97e0-44cbbdc6ff6d") : object "kube-system"/"original-pull-secret" not registered Jul 30 06:44:16.906734 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:16.906693 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs\") pod \"network-metrics-daemon-7tcdf\" (UID: \"2a6fa524-6766-4ecd-a86a-5f26f29a72b2\") " pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:16.906916 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:16.906813 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 06:44:16.906916 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:16.906888 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs podName:2a6fa524-6766-4ecd-a86a-5f26f29a72b2 nodeName:}" failed. No retries permitted until 2026-07-30 06:44:32.906866769 +0000 UTC m=+34.263870911 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs") pod "network-metrics-daemon-7tcdf" (UID: "2a6fa524-6766-4ecd-a86a-5f26f29a72b2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 06:44:17.007104 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:17.007070 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8src\" (UniqueName: \"kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src\") pod \"network-check-target-fr6kg\" (UID: \"b5281138-d6b6-4c65-a18f-6b153031868e\") " pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:17.007272 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:17.007234 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 06:44:17.007272 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:17.007252 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 06:44:17.007272 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:17.007272 2575 projected.go:196] Error preparing data for projected volume kube-api-access-l8src for pod openshift-network-diagnostics/network-check-target-fr6kg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 06:44:17.007457 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:17.007333 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src podName:b5281138-d6b6-4c65-a18f-6b153031868e nodeName:}" failed. No retries permitted until 2026-07-30 06:44:33.007314595 +0000 UTC m=+34.364318740 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-l8src" (UniqueName: "kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src") pod "network-check-target-fr6kg" (UID: "b5281138-d6b6-4c65-a18f-6b153031868e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 06:44:17.171538 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:17.171506 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:17.171714 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:17.171675 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:18.170455 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:18.170420 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:18.170924 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:18.170550 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:18.171008 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:18.170951 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:18.171067 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:18.171036 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:19.172297 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:19.172259 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:19.172761 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:19.172387 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:20.171409 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:20.171375 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:20.171409 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:20.171406 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:20.171599 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:20.171492 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:20.171637 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:20.171596 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:21.171862 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.171427 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:21.172462 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:21.172000 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:21.287150 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.287122 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7p8zb" event={"ID":"2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9","Type":"ContainerStarted","Data":"0fcf034d66ed7936c9c45f6646811c92625f3861a99c1946b5e353fcc0d60c72"} Jul 30 06:44:21.288522 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.288495 2575 generic.go:356] "Generic (PLEG): container finished" podID="c57d113e-75be-4b93-92f6-4c28e8a4714e" containerID="f6f629869870405286e3d998e31a751fdf273cf8328582be6a65a5a503564ff7" exitCode=0 Jul 30 06:44:21.288635 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.288564 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v7s2f" event={"ID":"c57d113e-75be-4b93-92f6-4c28e8a4714e","Type":"ContainerDied","Data":"f6f629869870405286e3d998e31a751fdf273cf8328582be6a65a5a503564ff7"} Jul 30 06:44:21.289954 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.289930 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" event={"ID":"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a","Type":"ContainerStarted","Data":"915aa0a003cb1556e4263c0d526887d3d92f9c85c31eff7af64d69a980fefb0c"} Jul 30 06:44:21.292193 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.292178 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 06:44:21.292463 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.292444 2575 generic.go:356] "Generic (PLEG): container finished" podID="8e9d780d-93f9-40cd-a0f1-9577b8096849" containerID="89b0eb3c7d5af8787761c657ece0b1b95d6d360089baddc809ee96343bc89827" exitCode=1 Jul 30 06:44:21.292527 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.292507 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" event={"ID":"8e9d780d-93f9-40cd-a0f1-9577b8096849","Type":"ContainerStarted","Data":"56adbb322a5cd1af16e44ec55d6ff0d1c7bec31f31b6b6052f5e016c5292e9b6"} Jul 30 06:44:21.292590 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.292575 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" event={"ID":"8e9d780d-93f9-40cd-a0f1-9577b8096849","Type":"ContainerDied","Data":"89b0eb3c7d5af8787761c657ece0b1b95d6d360089baddc809ee96343bc89827"} Jul 30 06:44:21.292630 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.292621 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" event={"ID":"8e9d780d-93f9-40cd-a0f1-9577b8096849","Type":"ContainerStarted","Data":"333718d6aca6da1ea60df917e2d2ff0927fdc06863d8db8d0841ca8ab854769c"} Jul 30 06:44:21.293730 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.293713 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xmcq2" event={"ID":"7adaf671-8982-4075-92cd-d190635a2fe7","Type":"ContainerStarted","Data":"d2f9ab78467ad121ccc091e652c36877d1fb590605299f5c67e8db70e6d81e55"} Jul 30 06:44:21.295102 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.295080 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" event={"ID":"c7ceee03-f5e3-40fe-93a7-073ee217ae14","Type":"ContainerStarted","Data":"81a412d14ad03a30f92370236059529a052f869fbf68449257fdc0cf320e629f"} Jul 30 06:44:21.296377 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.296356 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hdqsd" event={"ID":"7ce36c05-2aac-4717-8112-27dc9768ff02","Type":"ContainerStarted","Data":"cad8f33a4ddc047c7a82e95554aa30eac35f8025612abef3bce832a19c4e5373"} Jul 30 06:44:21.297648 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.297630 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h785z" event={"ID":"961a0a68-4f71-40b4-926a-d9d3fda38987","Type":"ContainerStarted","Data":"12dda92263fc0a3605bc2de3fe4afb2ce0d1f861c057feb7eba666b939a569c2"} Jul 30 06:44:21.301936 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.301898 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-185.ec2.internal" podStartSLOduration=21.301886066 podStartE2EDuration="21.301886066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 06:44:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:44:04.267394701 +0000 UTC m=+5.624398883" watchObservedRunningTime="2026-07-30 06:44:21.301886066 +0000 UTC m=+22.658890227" Jul 30 06:44:21.302277 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.302254 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7p8zb" podStartSLOduration=3.781616406 podStartE2EDuration="22.302249155s" totalImagesPullingTime="18.520632749s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 06:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:44:21.30173989 +0000 UTC m=+22.658744051" watchObservedRunningTime="2026-07-30 06:44:21.302249155 +0000 UTC m=+22.659253316" Jul 30 06:44:21.317629 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.317597 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jqjxs" podStartSLOduration=3.745855622 podStartE2EDuration="22.317585127s" totalImagesPullingTime="18.571729505s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 06:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:44:21.317166843 +0000 UTC m=+22.674171004" watchObservedRunningTime="2026-07-30 06:44:21.317585127 +0000 UTC m=+22.674589289" Jul 30 06:44:21.330886 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.330850 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hdqsd" podStartSLOduration=3.785638951 podStartE2EDuration="22.330836326s" totalImagesPullingTime="18.545197375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:44:21.330503591 +0000 UTC m=+22.687507752" watchObservedRunningTime="2026-07-30 06:44:21.330836326 +0000 UTC m=+22.687840487" Jul 30 06:44:21.358039 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.357997 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-h785z" podStartSLOduration=3.785262682 podStartE2EDuration="22.357984811s" totalImagesPullingTime="18.572722129s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 06:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:44:21.356261885 +0000 UTC m=+22.713266048" watchObservedRunningTime="2026-07-30 06:44:21.357984811 +0000 UTC m=+22.714988972" Jul 30 06:44:21.377430 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:21.377394 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xmcq2" podStartSLOduration=3.856887158 podStartE2EDuration="22.377380162s" totalImagesPullingTime="18.520493004s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 06:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:44:21.377377018 +0000 UTC m=+22.734381180" watchObservedRunningTime="2026-07-30 06:44:21.377380162 +0000 UTC m=+22.734384314" Jul 30 06:44:22.101962 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:22.101934 2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 30 06:44:22.135455 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:22.135350 2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T06:44:22.10195569Z","UUID":"b15e26b0-e2bc-45a9-9729-630ce893a1fc","Handler":null,"Name":"","Endpoint":""} Jul 30 06:44:22.137147 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:22.137128 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 30 06:44:22.137254 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:22.137168 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 30 06:44:22.171186 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:22.171156 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:22.171340 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:22.171286 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:22.171340 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:22.171318 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:22.171448 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:22.171389 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:22.303136 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:22.303063 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 06:44:22.303563 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:22.303489 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" event={"ID":"8e9d780d-93f9-40cd-a0f1-9577b8096849","Type":"ContainerStarted","Data":"8c53c42909a4a918ccf0573d57f1913761e109f3b9ed2ffa62f215faa80f3ec2"} Jul 30 06:44:22.303563 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:22.303560 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" event={"ID":"8e9d780d-93f9-40cd-a0f1-9577b8096849","Type":"ContainerStarted","Data":"4a5efc376230e253bd4df074f879d38268cf1bf02d7dafed790254e0bf05c0a4"} Jul 30 06:44:22.303724 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:22.303605 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" event={"ID":"8e9d780d-93f9-40cd-a0f1-9577b8096849","Type":"ContainerStarted","Data":"b457a6d96ac3e248d6ef472c23d86c370da9ca293a4b84cb69c22aaec1ccf5c5"} Jul 30 06:44:22.305135 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:22.305108 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kkrfh" event={"ID":"03374672-1c6c-4996-ae88-a5fa11c92a57","Type":"ContainerStarted","Data":"dee9b57caea18dc478e92740380e7b08a00438760fbc6a683a6861a2538cb467"} Jul 30 06:44:22.307072 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:22.307012 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" event={"ID":"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a","Type":"ContainerStarted","Data":"66aa85eadf2ad42e4a6733dfc2cd8cdcf3156b3a07ef4e98dcd0264701456d76"} Jul 30 06:44:22.319311 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:22.319259 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kkrfh" podStartSLOduration=4.715229554 podStartE2EDuration="23.319245963s" totalImagesPullingTime="18.604016409s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 06:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:44:22.318844702 +0000 UTC m=+23.675848864" watchObservedRunningTime="2026-07-30 06:44:22.319245963 +0000 UTC m=+23.676250125" Jul 30 06:44:23.170981 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:23.170747 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:23.171113 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:23.171083 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:23.311862 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:23.311821 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" event={"ID":"a6b4d2f1-d7eb-4577-be6e-6b1bd598ff2a","Type":"ContainerStarted","Data":"e9c122a4df56c49162f2af59d362dd2533e3117112e601af2e07b984eb1da121"} Jul 30 06:44:23.330590 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:23.330509 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vkj6s" podStartSLOduration=3.742674474 podStartE2EDuration="24.330493816s" totalImagesPullingTime="20.587819342s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 06:43:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:44:23.329200894 +0000 UTC m=+24.686205057" watchObservedRunningTime="2026-07-30 06:44:23.330493816 +0000 UTC m=+24.687497977" Jul 30 06:44:24.171299 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:24.170830 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:24.171299 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:24.170976 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:24.171299 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:24.171023 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:24.171299 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:24.171126 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:24.317248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:24.317212 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 06:44:24.317725 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:24.317692 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" event={"ID":"8e9d780d-93f9-40cd-a0f1-9577b8096849","Type":"ContainerStarted","Data":"5663490d0afef06f2016f62e0ccd8c6c705c3faf72a02c9f95637092f1e16114"} Jul 30 06:44:24.519002 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:24.518922 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hdqsd" Jul 30 06:44:24.519641 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:24.519617 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hdqsd" Jul 30 06:44:25.171112 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:25.171083 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:25.171293 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:25.171231 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:25.320295 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:25.320264 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hdqsd" Jul 30 06:44:25.320932 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:25.320830 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hdqsd" Jul 30 06:44:26.171084 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:26.170905 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:26.171221 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:26.170948 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:26.171221 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:26.171171 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:26.171332 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:26.171241 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:26.327208 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:26.327170 2575 generic.go:356] "Generic (PLEG): container finished" podID="c57d113e-75be-4b93-92f6-4c28e8a4714e" containerID="56626809f264b4978795909f588692db87f5bd42ed28bb9967fe279d8588f9c2" exitCode=0 Jul 30 06:44:26.327869 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:26.327248 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v7s2f" event={"ID":"c57d113e-75be-4b93-92f6-4c28e8a4714e","Type":"ContainerDied","Data":"56626809f264b4978795909f588692db87f5bd42ed28bb9967fe279d8588f9c2"} Jul 30 06:44:26.330880 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:26.330865 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 06:44:26.331191 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:26.331171 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" event={"ID":"8e9d780d-93f9-40cd-a0f1-9577b8096849","Type":"ContainerStarted","Data":"dfe9dd1a1be4eb02ede8c945d58da30d2c32a7a2cc76460bdbacae0c3e3f83d6"} Jul 30 06:44:26.331522 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:26.331502 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:26.331588 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:26.331569 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:26.331588 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:26.331579 2575 scope.go:117] "RemoveContainer" containerID="89b0eb3c7d5af8787761c657ece0b1b95d6d360089baddc809ee96343bc89827" Jul 30 06:44:26.349266 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:26.349221 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:26.349646 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:26.349630 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:27.171049 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:27.171024 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:27.171178 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:27.171125 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:27.335763 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:27.335677 2575 generic.go:356] "Generic (PLEG): container finished" podID="c57d113e-75be-4b93-92f6-4c28e8a4714e" containerID="4317acabc7d271a489cd9d413b5b37e1cb63a053eaa76f9a3bb8afd3b5af1f7a" exitCode=0 Jul 30 06:44:27.336147 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:27.335765 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v7s2f" event={"ID":"c57d113e-75be-4b93-92f6-4c28e8a4714e","Type":"ContainerDied","Data":"4317acabc7d271a489cd9d413b5b37e1cb63a053eaa76f9a3bb8afd3b5af1f7a"} Jul 30 06:44:27.339997 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:27.339978 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 06:44:27.340334 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:27.340312 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" event={"ID":"8e9d780d-93f9-40cd-a0f1-9577b8096849","Type":"ContainerStarted","Data":"a89afd396f7706284259f57b680191584cfe0a8087c93103bd3509102ba15fbd"} Jul 30 06:44:27.340420 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:27.340403 2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 30 06:44:27.385346 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:27.385297 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" podStartSLOduration=9.760099603 podStartE2EDuration="28.385281067s" totalImagesPullingTime="18.625181464s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 06:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:44:27.384893438 +0000 UTC m=+28.741897601" watchObservedRunningTime="2026-07-30 06:44:27.385281067 +0000 UTC m=+28.742285229" Jul 30 06:44:27.821751 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:27.821720 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sc2rl"] Jul 30 06:44:27.821907 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:27.821873 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:27.822009 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:27.821986 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:27.826157 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:27.826126 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7tcdf"] Jul 30 06:44:27.827012 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:27.826193 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fr6kg"] Jul 30 06:44:27.827012 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:27.826283 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:27.827012 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:27.826375 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:27.827012 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:27.826449 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:27.827012 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:27.826535 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:27.893833 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:27.893803 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:28.344793 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:28.344711 2575 generic.go:356] "Generic (PLEG): container finished" podID="c57d113e-75be-4b93-92f6-4c28e8a4714e" containerID="08bf40c30f37cd7b677302fcb3583e0bc5cd1691f4ccbb556903f3bc68723ed7" exitCode=0 Jul 30 06:44:28.345202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:28.344809 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v7s2f" event={"ID":"c57d113e-75be-4b93-92f6-4c28e8a4714e","Type":"ContainerDied","Data":"08bf40c30f37cd7b677302fcb3583e0bc5cd1691f4ccbb556903f3bc68723ed7"} Jul 30 06:44:29.172622 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:29.172404 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:29.173294 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:29.173254 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:29.173736 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:29.173715 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:29.173992 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:29.173976 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:29.174188 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:29.174167 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:29.174306 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:29.174283 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:31.171380 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:31.171345 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:31.172074 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:31.171387 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:31.172074 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:31.171350 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:31.172074 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:31.171480 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:31.172074 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:31.171616 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:31.172074 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:31.171735 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:32.819751 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:32.819716 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret\") pod \"global-pull-secret-syncer-sc2rl\" (UID: \"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d\") " pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:32.820236 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:32.819850 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 30 06:44:32.820236 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:32.819928 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret podName:dd5d594e-9943-4d46-97e0-44cbbdc6ff6d nodeName:}" failed. No retries permitted until 2026-07-30 06:45:04.819907504 +0000 UTC m=+66.176911646 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret") pod "global-pull-secret-syncer-sc2rl" (UID: "dd5d594e-9943-4d46-97e0-44cbbdc6ff6d") : object "kube-system"/"original-pull-secret" not registered Jul 30 06:44:32.919997 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:32.919976 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs\") pod \"network-metrics-daemon-7tcdf\" (UID: \"2a6fa524-6766-4ecd-a86a-5f26f29a72b2\") " pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:32.920132 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:32.920116 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 06:44:32.920188 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:32.920179 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs podName:2a6fa524-6766-4ecd-a86a-5f26f29a72b2 nodeName:}" failed. No retries permitted until 2026-07-30 06:45:04.920165774 +0000 UTC m=+66.277169918 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs") pod "network-metrics-daemon-7tcdf" (UID: "2a6fa524-6766-4ecd-a86a-5f26f29a72b2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 30 06:44:33.020861 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:33.020833 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8src\" (UniqueName: \"kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src\") pod \"network-check-target-fr6kg\" (UID: \"b5281138-d6b6-4c65-a18f-6b153031868e\") " pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:33.021054 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:33.021035 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 30 06:44:33.021054 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:33.021059 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 30 06:44:33.021180 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:33.021070 2575 projected.go:196] Error preparing data for projected volume kube-api-access-l8src for pod openshift-network-diagnostics/network-check-target-fr6kg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 06:44:33.021180 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:33.021119 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src podName:b5281138-d6b6-4c65-a18f-6b153031868e nodeName:}" failed. No retries permitted until 2026-07-30 06:45:05.021104987 +0000 UTC m=+66.378109126 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-l8src" (UniqueName: "kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src") pod "network-check-target-fr6kg" (UID: "b5281138-d6b6-4c65-a18f-6b153031868e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 30 06:44:33.170882 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:33.170855 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:33.171078 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:33.170862 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:33.171078 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:33.170967 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:44:33.171078 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:33.170862 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:33.171078 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:33.171022 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fr6kg" podUID="b5281138-d6b6-4c65-a18f-6b153031868e" Jul 30 06:44:33.171286 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:33.171129 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sc2rl" podUID="dd5d594e-9943-4d46-97e0-44cbbdc6ff6d" Jul 30 06:44:33.359732 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:33.359701 2575 generic.go:356] "Generic (PLEG): container finished" podID="c57d113e-75be-4b93-92f6-4c28e8a4714e" containerID="fe2bcce4304fcb7513a8b565354d75d84fd1a4f91a14d243307faa047aed488a" exitCode=0 Jul 30 06:44:33.359921 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:33.359757 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v7s2f" event={"ID":"c57d113e-75be-4b93-92f6-4c28e8a4714e","Type":"ContainerDied","Data":"fe2bcce4304fcb7513a8b565354d75d84fd1a4f91a14d243307faa047aed488a"} Jul 30 06:44:34.364624 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.364593 2575 generic.go:356] "Generic (PLEG): container finished" podID="c57d113e-75be-4b93-92f6-4c28e8a4714e" containerID="4adf0f471d2ede7b5b2aea4b0d92a48d4cb85698bd5deb0e8155a945d46cafac" exitCode=0 Jul 30 06:44:34.365191 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.364670 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v7s2f" event={"ID":"c57d113e-75be-4b93-92f6-4c28e8a4714e","Type":"ContainerDied","Data":"4adf0f471d2ede7b5b2aea4b0d92a48d4cb85698bd5deb0e8155a945d46cafac"} Jul 30 06:44:34.462609 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.462444 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-185.ec2.internal" event="NodeReady" Jul 30 06:44:34.462763 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.462750 2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 30 06:44:34.495209 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.495186 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77d84d85c6-wscbq"] Jul 30 06:44:34.508976 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.508947 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-svjsc"] Jul 30 06:44:34.509182 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.509012 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.513178 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.513157 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 30 06:44:34.513300 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.513157 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f2g8l\"" Jul 30 06:44:34.513300 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.513158 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 30 06:44:34.513300 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.513157 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 30 06:44:34.524248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.522881 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pvswz"] Jul 30 06:44:34.524248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.522999 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:44:34.524248 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.524045 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 30 06:44:34.526309 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.525702 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 30 06:44:34.526309 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.525799 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-mwlhp\"" Jul 30 06:44:34.526309 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.525805 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 30 06:44:34.533815 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.533796 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bnkvv"] Jul 30 06:44:34.533898 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.533892 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:34.536634 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.536618 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cvd6x\"" Jul 30 06:44:34.536789 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.536775 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 30 06:44:34.537111 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.537098 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 30 06:44:34.554288 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.554253 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-svjsc"] Jul 30 06:44:34.554382 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.554260 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bnkvv" Jul 30 06:44:34.554445 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.554341 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77d84d85c6-wscbq"] Jul 30 06:44:34.554505 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.554459 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pvswz"] Jul 30 06:44:34.554505 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.554486 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bnkvv"] Jul 30 06:44:34.556776 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.556759 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 30 06:44:34.556922 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.556818 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 30 06:44:34.556922 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.556763 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 30 06:44:34.556922 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.556851 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-m9kjz\"" Jul 30 06:44:34.633584 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.633555 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-bound-sa-token\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.633765 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.633588 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/52df3cd9-75cc-47c9-b997-48295db78d16-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-svjsc\" (UID: \"52df3cd9-75cc-47c9-b997-48295db78d16\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:44:34.633765 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.633612 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.633765 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.633635 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1ba0bcb-7d3d-447e-8272-4910b15167d3-installation-pull-secrets\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.633765 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.633681 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e85b5491-8587-430a-ac24-a541065ee2da-tmp-dir\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:34.633765 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.633708 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7fjf\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-kube-api-access-f7fjf\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.633765 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.633729 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-svjsc\" (UID: \"52df3cd9-75cc-47c9-b997-48295db78d16\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:44:34.633989 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.633781 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1ba0bcb-7d3d-447e-8272-4910b15167d3-image-registry-private-configuration\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.633989 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.633804 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1ba0bcb-7d3d-447e-8272-4910b15167d3-trusted-ca\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.633989 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.633856 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1ba0bcb-7d3d-447e-8272-4910b15167d3-ca-trust-extracted\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.633989 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.633874 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:34.633989 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.633887 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/510fa8de-adc5-476f-bb43-50233a5f291b-cert\") pod \"ingress-canary-bnkvv\" (UID: \"510fa8de-adc5-476f-bb43-50233a5f291b\") " pod="openshift-ingress-canary/ingress-canary-bnkvv" Jul 30 06:44:34.633989 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.633929 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-certificates\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.633989 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.633961 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e85b5491-8587-430a-ac24-a541065ee2da-config-volume\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:34.633989 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.633985 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw427\" (UniqueName: \"kubernetes.io/projected/e85b5491-8587-430a-ac24-a541065ee2da-kube-api-access-kw427\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:34.634219 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.634002 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25lqb\" (UniqueName: \"kubernetes.io/projected/510fa8de-adc5-476f-bb43-50233a5f291b-kube-api-access-25lqb\") pod \"ingress-canary-bnkvv\" (UID: \"510fa8de-adc5-476f-bb43-50233a5f291b\") " pod="openshift-ingress-canary/ingress-canary-bnkvv" Jul 30 06:44:34.651886 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.651854 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:34.734973 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.734941 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1ba0bcb-7d3d-447e-8272-4910b15167d3-image-registry-private-configuration\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.735130 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.734979 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1ba0bcb-7d3d-447e-8272-4910b15167d3-trusted-ca\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.735130 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.735036 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1ba0bcb-7d3d-447e-8272-4910b15167d3-ca-trust-extracted\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.735239 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.735196 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:34.735338 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.735235 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/510fa8de-adc5-476f-bb43-50233a5f291b-cert\") pod \"ingress-canary-bnkvv\" (UID: \"510fa8de-adc5-476f-bb43-50233a5f291b\") " pod="openshift-ingress-canary/ingress-canary-bnkvv" Jul 30 06:44:34.735338 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.735292 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-certificates\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.735338 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.735323 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e85b5491-8587-430a-ac24-a541065ee2da-config-volume\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:34.735473 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.735354 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw427\" (UniqueName: \"kubernetes.io/projected/e85b5491-8587-430a-ac24-a541065ee2da-kube-api-access-kw427\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:34.735473 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:34.735356 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 06:44:34.735473 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.735377 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25lqb\" (UniqueName: \"kubernetes.io/projected/510fa8de-adc5-476f-bb43-50233a5f291b-kube-api-access-25lqb\") pod \"ingress-canary-bnkvv\" (UID: \"510fa8de-adc5-476f-bb43-50233a5f291b\") " pod="openshift-ingress-canary/ingress-canary-bnkvv" Jul 30 06:44:34.735473 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:34.735427 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls podName:e85b5491-8587-430a-ac24-a541065ee2da nodeName:}" failed. No retries permitted until 2026-07-30 06:44:35.235408752 +0000 UTC m=+36.592412895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls") pod "dns-default-pvswz" (UID: "e85b5491-8587-430a-ac24-a541065ee2da") : secret "dns-default-metrics-tls" not found Jul 30 06:44:34.735808 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.735789 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-bound-sa-token\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.735898 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.735823 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/52df3cd9-75cc-47c9-b997-48295db78d16-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-svjsc\" (UID: \"52df3cd9-75cc-47c9-b997-48295db78d16\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:44:34.735898 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.735851 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.735898 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.735880 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1ba0bcb-7d3d-447e-8272-4910b15167d3-installation-pull-secrets\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.735898 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.735906 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e85b5491-8587-430a-ac24-a541065ee2da-tmp-dir\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:34.736081 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.735935 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7fjf\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-kube-api-access-f7fjf\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.736081 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.735962 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-svjsc\" (UID: \"52df3cd9-75cc-47c9-b997-48295db78d16\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:44:34.736160 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:34.736083 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 06:44:34.736160 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.736097 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:34.736160 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.736108 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e85b5491-8587-430a-ac24-a541065ee2da-config-volume\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:34.736160 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:34.736150 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert podName:52df3cd9-75cc-47c9-b997-48295db78d16 nodeName:}" failed. No retries permitted until 2026-07-30 06:44:35.236133587 +0000 UTC m=+36.593137727 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-svjsc" (UID: "52df3cd9-75cc-47c9-b997-48295db78d16") : secret "networking-console-plugin-cert" not found Jul 30 06:44:34.736382 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.736358 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e85b5491-8587-430a-ac24-a541065ee2da-tmp-dir\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:34.736678 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:34.736632 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 06:44:34.736788 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:34.736698 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77d84d85c6-wscbq: secret "image-registry-tls" not found Jul 30 06:44:34.736788 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:34.736741 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls podName:f1ba0bcb-7d3d-447e-8272-4910b15167d3 nodeName:}" failed. No retries permitted until 2026-07-30 06:44:35.236727619 +0000 UTC m=+36.593731759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls") pod "image-registry-77d84d85c6-wscbq" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3") : secret "image-registry-tls" not found Jul 30 06:44:34.736910 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.736893 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/52df3cd9-75cc-47c9-b997-48295db78d16-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-svjsc\" (UID: \"52df3cd9-75cc-47c9-b997-48295db78d16\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:44:34.739641 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.739622 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/510fa8de-adc5-476f-bb43-50233a5f291b-cert\") pod \"ingress-canary-bnkvv\" (UID: \"510fa8de-adc5-476f-bb43-50233a5f291b\") " pod="openshift-ingress-canary/ingress-canary-bnkvv" Jul 30 06:44:34.741273 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.741252 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1ba0bcb-7d3d-447e-8272-4910b15167d3-ca-trust-extracted\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.741509 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.741492 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-certificates\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.741577 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.741543 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1ba0bcb-7d3d-447e-8272-4910b15167d3-installation-pull-secrets\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.741577 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.741555 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1ba0bcb-7d3d-447e-8272-4910b15167d3-image-registry-private-configuration\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.741880 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.741857 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1ba0bcb-7d3d-447e-8272-4910b15167d3-trusted-ca\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.745006 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.744987 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25lqb\" (UniqueName: \"kubernetes.io/projected/510fa8de-adc5-476f-bb43-50233a5f291b-kube-api-access-25lqb\") pod \"ingress-canary-bnkvv\" (UID: \"510fa8de-adc5-476f-bb43-50233a5f291b\") " pod="openshift-ingress-canary/ingress-canary-bnkvv" Jul 30 06:44:34.745554 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.745535 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7fjf\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-kube-api-access-f7fjf\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.745679 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.745640 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-bound-sa-token\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:34.745848 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.745830 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw427\" (UniqueName: \"kubernetes.io/projected/e85b5491-8587-430a-ac24-a541065ee2da-kube-api-access-kw427\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:34.836681 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.836621 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:34.862766 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.862741 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bnkvv" Jul 30 06:44:34.938251 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:34.938220 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:35.009406 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.009375 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bnkvv"] Jul 30 06:44:35.013041 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:44:35.013011 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod510fa8de_adc5_476f_bb43_50233a5f291b.slice/crio-344d75b593099b14720f79a485ee3c85bf3c5083e852c84e427a7664524e17ed WatchSource:0}: Error finding container 344d75b593099b14720f79a485ee3c85bf3c5083e852c84e427a7664524e17ed: Status 404 returned error can't find the container with id 344d75b593099b14720f79a485ee3c85bf3c5083e852c84e427a7664524e17ed Jul 30 06:44:35.170750 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.170717 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:44:35.170750 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.170741 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:44:35.170999 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.170722 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:44:35.173724 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.173630 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 30 06:44:35.173724 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.173646 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8tfrn\"" Jul 30 06:44:35.173927 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.173858 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 30 06:44:35.173983 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.173932 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 30 06:44:35.174040 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.173984 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rnxxz\"" Jul 30 06:44:35.174749 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.174731 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 30 06:44:35.239121 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.239090 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:35.239264 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.239128 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-svjsc\" (UID: \"52df3cd9-75cc-47c9-b997-48295db78d16\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:44:35.239264 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:35.239230 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 06:44:35.239264 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:35.239237 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 06:44:35.239264 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:35.239259 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77d84d85c6-wscbq: secret "image-registry-tls" not found Jul 30 06:44:35.239447 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:35.239285 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert podName:52df3cd9-75cc-47c9-b997-48295db78d16 nodeName:}" failed. No retries permitted until 2026-07-30 06:44:36.239271299 +0000 UTC m=+37.596275438 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-svjsc" (UID: "52df3cd9-75cc-47c9-b997-48295db78d16") : secret "networking-console-plugin-cert" not found Jul 30 06:44:35.239447 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:35.239314 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls podName:f1ba0bcb-7d3d-447e-8272-4910b15167d3 nodeName:}" failed. No retries permitted until 2026-07-30 06:44:36.239300841 +0000 UTC m=+37.596304984 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls") pod "image-registry-77d84d85c6-wscbq" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3") : secret "image-registry-tls" not found Jul 30 06:44:35.239447 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.239350 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:35.239546 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:35.239451 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 06:44:35.239546 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:35.239484 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls podName:e85b5491-8587-430a-ac24-a541065ee2da nodeName:}" failed. No retries permitted until 2026-07-30 06:44:36.239476436 +0000 UTC m=+37.596480578 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls") pod "dns-default-pvswz" (UID: "e85b5491-8587-430a-ac24-a541065ee2da") : secret "dns-default-metrics-tls" not found Jul 30 06:44:35.370804 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.370754 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v7s2f" event={"ID":"c57d113e-75be-4b93-92f6-4c28e8a4714e","Type":"ContainerStarted","Data":"4d36075d5aa26395101d7055507442c25864567273d1aac9ff20e7a6dcca4e0f"} Jul 30 06:44:35.371832 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.371808 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bnkvv" event={"ID":"510fa8de-adc5-476f-bb43-50233a5f291b","Type":"ContainerStarted","Data":"344d75b593099b14720f79a485ee3c85bf3c5083e852c84e427a7664524e17ed"} Jul 30 06:44:35.393110 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.391288 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-v7s2f" podStartSLOduration=7.570284555 podStartE2EDuration="36.391268729s" totalImagesPullingTime="28.820984174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 06:43:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:44:35.390360351 +0000 UTC m=+36.747364512" watchObservedRunningTime="2026-07-30 06:44:35.391268729 +0000 UTC m=+36.748272890" Jul 30 06:44:35.513294 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.513212 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:35.566383 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:35.566342 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:36.248340 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:36.248302 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:36.248511 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:36.248368 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-svjsc\" (UID: \"52df3cd9-75cc-47c9-b997-48295db78d16\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:44:36.248511 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:36.248441 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:36.248511 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:36.248480 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 06:44:36.248511 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:36.248503 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77d84d85c6-wscbq: secret "image-registry-tls" not found Jul 30 06:44:36.248698 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:36.248562 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 06:44:36.248698 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:36.248576 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 06:44:36.248698 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:36.248585 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls podName:f1ba0bcb-7d3d-447e-8272-4910b15167d3 nodeName:}" failed. No retries permitted until 2026-07-30 06:44:38.248564412 +0000 UTC m=+39.605568557 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls") pod "image-registry-77d84d85c6-wscbq" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3") : secret "image-registry-tls" not found Jul 30 06:44:36.248698 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:36.248605 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert podName:52df3cd9-75cc-47c9-b997-48295db78d16 nodeName:}" failed. No retries permitted until 2026-07-30 06:44:38.248594226 +0000 UTC m=+39.605598375 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-svjsc" (UID: "52df3cd9-75cc-47c9-b997-48295db78d16") : secret "networking-console-plugin-cert" not found Jul 30 06:44:36.248698 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:36.248622 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls podName:e85b5491-8587-430a-ac24-a541065ee2da nodeName:}" failed. No retries permitted until 2026-07-30 06:44:38.248613205 +0000 UTC m=+39.605617349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls") pod "dns-default-pvswz" (UID: "e85b5491-8587-430a-ac24-a541065ee2da") : secret "dns-default-metrics-tls" not found Jul 30 06:44:37.417804 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:37.417560 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:37.501743 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:37.501712 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:37.596525 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:37.596488 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:37.696127 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:37.696039 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:37.762366 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:37.762316 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:38.265962 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:38.265877 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:38.265962 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:38.265937 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:38.266184 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:38.265999 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-svjsc\" (UID: \"52df3cd9-75cc-47c9-b997-48295db78d16\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:44:38.266184 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:38.266049 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 06:44:38.266184 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:38.266080 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 06:44:38.266184 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:38.266047 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 06:44:38.266184 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:38.266121 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls podName:e85b5491-8587-430a-ac24-a541065ee2da nodeName:}" failed. No retries permitted until 2026-07-30 06:44:42.266101494 +0000 UTC m=+43.623105650 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls") pod "dns-default-pvswz" (UID: "e85b5491-8587-430a-ac24-a541065ee2da") : secret "dns-default-metrics-tls" not found Jul 30 06:44:38.266184 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:38.266123 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77d84d85c6-wscbq: secret "image-registry-tls" not found Jul 30 06:44:38.266184 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:38.266139 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert podName:52df3cd9-75cc-47c9-b997-48295db78d16 nodeName:}" failed. No retries permitted until 2026-07-30 06:44:42.266130152 +0000 UTC m=+43.623134308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-svjsc" (UID: "52df3cd9-75cc-47c9-b997-48295db78d16") : secret "networking-console-plugin-cert" not found Jul 30 06:44:38.266184 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:38.266155 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls podName:f1ba0bcb-7d3d-447e-8272-4910b15167d3 nodeName:}" failed. No retries permitted until 2026-07-30 06:44:42.266146672 +0000 UTC m=+43.623150814 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls") pod "image-registry-77d84d85c6-wscbq" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3") : secret "image-registry-tls" not found Jul 30 06:44:38.381332 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:38.381299 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bnkvv" event={"ID":"510fa8de-adc5-476f-bb43-50233a5f291b","Type":"ContainerStarted","Data":"5a1ba51d8730c532aa26b33e588baf29571e7342820917c7ad88358a8c057a2e"} Jul 30 06:44:38.385686 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:38.385627 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:38.396099 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:38.396056 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bnkvv" podStartSLOduration=1.5236603180000001 podStartE2EDuration="4.396040232s" totalImagesPullingTime="2.872379914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:44:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:44:38.395733795 +0000 UTC m=+39.752737958" watchObservedRunningTime="2026-07-30 06:44:38.396040232 +0000 UTC m=+39.753044393" Jul 30 06:44:38.486293 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:38.486263 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:38.758887 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:38.758848 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:38.859097 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:38.859065 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:39.099895 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:39.099805 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:39.148579 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:39.148551 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:39.577625 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:39.577589 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:39.636839 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:39.636802 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:39.988376 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:39.988340 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:40.088701 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:40.088668 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:40.740416 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:40.740367 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:40.841689 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:40.841640 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:41.181631 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:41.181603 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:41.239943 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:41.239906 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:42.296737 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:42.296704 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:42.297094 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:42.296767 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:42.297094 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:42.296803 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-svjsc\" (UID: \"52df3cd9-75cc-47c9-b997-48295db78d16\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:44:42.297094 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:42.296866 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 06:44:42.297094 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:42.296920 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 06:44:42.297094 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:42.296926 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 06:44:42.297094 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:42.296945 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77d84d85c6-wscbq: secret "image-registry-tls" not found Jul 30 06:44:42.297094 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:42.296933 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls podName:e85b5491-8587-430a-ac24-a541065ee2da nodeName:}" failed. No retries permitted until 2026-07-30 06:44:50.296916641 +0000 UTC m=+51.653920781 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls") pod "dns-default-pvswz" (UID: "e85b5491-8587-430a-ac24-a541065ee2da") : secret "dns-default-metrics-tls" not found Jul 30 06:44:42.297094 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:42.296988 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert podName:52df3cd9-75cc-47c9-b997-48295db78d16 nodeName:}" failed. No retries permitted until 2026-07-30 06:44:50.296972998 +0000 UTC m=+51.653977140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-svjsc" (UID: "52df3cd9-75cc-47c9-b997-48295db78d16") : secret "networking-console-plugin-cert" not found Jul 30 06:44:42.297094 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:42.297005 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls podName:f1ba0bcb-7d3d-447e-8272-4910b15167d3 nodeName:}" failed. No retries permitted until 2026-07-30 06:44:50.296999438 +0000 UTC m=+51.654003578 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls") pod "image-registry-77d84d85c6-wscbq" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3") : secret "image-registry-tls" not found Jul 30 06:44:49.724249 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:49.724214 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:49.779118 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:49.779083 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:50.354213 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:50.354179 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:44:50.354477 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:50.354241 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:44:50.354477 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:50.354272 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-svjsc\" (UID: \"52df3cd9-75cc-47c9-b997-48295db78d16\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:44:50.354477 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:50.354350 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 06:44:50.354477 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:50.354373 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 06:44:50.354477 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:50.354389 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 06:44:50.354477 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:50.354403 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77d84d85c6-wscbq: secret "image-registry-tls" not found Jul 30 06:44:50.354477 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:50.354425 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls podName:e85b5491-8587-430a-ac24-a541065ee2da nodeName:}" failed. No retries permitted until 2026-07-30 06:45:06.354402869 +0000 UTC m=+67.711407011 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls") pod "dns-default-pvswz" (UID: "e85b5491-8587-430a-ac24-a541065ee2da") : secret "dns-default-metrics-tls" not found Jul 30 06:44:50.354477 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:50.354441 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert podName:52df3cd9-75cc-47c9-b997-48295db78d16 nodeName:}" failed. No retries permitted until 2026-07-30 06:45:06.354434627 +0000 UTC m=+67.711438768 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-svjsc" (UID: "52df3cd9-75cc-47c9-b997-48295db78d16") : secret "networking-console-plugin-cert" not found Jul 30 06:44:50.354477 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:44:50.354465 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls podName:f1ba0bcb-7d3d-447e-8272-4910b15167d3 nodeName:}" failed. No retries permitted until 2026-07-30 06:45:06.35445174 +0000 UTC m=+67.711455895 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls") pod "image-registry-77d84d85c6-wscbq" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3") : secret "image-registry-tls" not found Jul 30 06:44:52.049414 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:52.049374 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:52.150318 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:52.150283 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:54.908529 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:54.908488 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:54.958400 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:54.958365 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:55.187786 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:55.187697 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:55.288739 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:55.288701 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:59.358223 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:59.358194 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-84tq9" Jul 30 06:44:59.511120 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:59.511075 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:59.563108 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:59.563067 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:44:59.718851 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:44:59.718819 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" limit=1024 Jul 30 06:45:04.860182 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:04.860139 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret\") pod \"global-pull-secret-syncer-sc2rl\" (UID: \"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d\") " pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:45:04.862820 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:04.862804 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 30 06:45:04.874231 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:04.874206 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dd5d594e-9943-4d46-97e0-44cbbdc6ff6d-original-pull-secret\") pod \"global-pull-secret-syncer-sc2rl\" (UID: \"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d\") " pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:45:04.882267 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:04.882239 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sc2rl" Jul 30 06:45:04.961647 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:04.961441 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs\") pod \"network-metrics-daemon-7tcdf\" (UID: \"2a6fa524-6766-4ecd-a86a-5f26f29a72b2\") " pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:45:04.964608 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:04.964584 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 30 06:45:04.971620 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:45:04.971592 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 30 06:45:04.971743 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:45:04.971688 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs podName:2a6fa524-6766-4ecd-a86a-5f26f29a72b2 nodeName:}" failed. No retries permitted until 2026-07-30 06:46:08.971670079 +0000 UTC m=+130.328674234 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs") pod "network-metrics-daemon-7tcdf" (UID: "2a6fa524-6766-4ecd-a86a-5f26f29a72b2") : secret "metrics-daemon-secret" not found Jul 30 06:45:05.024262 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:05.024232 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sc2rl"] Jul 30 06:45:05.027512 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:45:05.027485 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd5d594e_9943_4d46_97e0_44cbbdc6ff6d.slice/crio-d84c8a38b606b15770e0b5f146a8feb25519dc5af594e76f77cf68aabe9148c7 WatchSource:0}: Error finding container d84c8a38b606b15770e0b5f146a8feb25519dc5af594e76f77cf68aabe9148c7: Status 404 returned error can't find the container with id d84c8a38b606b15770e0b5f146a8feb25519dc5af594e76f77cf68aabe9148c7 Jul 30 06:45:05.062060 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:05.062035 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8src\" (UniqueName: \"kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src\") pod \"network-check-target-fr6kg\" (UID: \"b5281138-d6b6-4c65-a18f-6b153031868e\") " pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:45:05.064830 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:05.064804 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 30 06:45:05.075235 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:05.075219 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 30 06:45:05.085686 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:05.085635 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8src\" (UniqueName: \"kubernetes.io/projected/b5281138-d6b6-4c65-a18f-6b153031868e-kube-api-access-l8src\") pod \"network-check-target-fr6kg\" (UID: \"b5281138-d6b6-4c65-a18f-6b153031868e\") " pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:45:05.195674 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:05.195633 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rnxxz\"" Jul 30 06:45:05.203645 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:05.203629 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:45:05.325160 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:05.325128 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fr6kg"] Jul 30 06:45:05.328299 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:45:05.328254 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5281138_d6b6_4c65_a18f_6b153031868e.slice/crio-9203cd7e7dfe1d0e88cb3776cc0018ad596210d95130c1df2ce6c2cdd1d222bb WatchSource:0}: Error finding container 9203cd7e7dfe1d0e88cb3776cc0018ad596210d95130c1df2ce6c2cdd1d222bb: Status 404 returned error can't find the container with id 9203cd7e7dfe1d0e88cb3776cc0018ad596210d95130c1df2ce6c2cdd1d222bb Jul 30 06:45:05.442580 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:05.442535 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fr6kg" event={"ID":"b5281138-d6b6-4c65-a18f-6b153031868e","Type":"ContainerStarted","Data":"9203cd7e7dfe1d0e88cb3776cc0018ad596210d95130c1df2ce6c2cdd1d222bb"} Jul 30 06:45:05.443632 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:05.443610 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sc2rl" event={"ID":"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d","Type":"ContainerStarted","Data":"d84c8a38b606b15770e0b5f146a8feb25519dc5af594e76f77cf68aabe9148c7"} Jul 30 06:45:06.372969 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:06.372625 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:45:06.372969 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:06.372726 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:45:06.372969 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:06.372754 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-svjsc\" (UID: \"52df3cd9-75cc-47c9-b997-48295db78d16\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:45:06.372969 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:45:06.372797 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 06:45:06.372969 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:45:06.372867 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 06:45:06.372969 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:45:06.372882 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls podName:e85b5491-8587-430a-ac24-a541065ee2da nodeName:}" failed. No retries permitted until 2026-07-30 06:45:38.372858564 +0000 UTC m=+99.729862710 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls") pod "dns-default-pvswz" (UID: "e85b5491-8587-430a-ac24-a541065ee2da") : secret "dns-default-metrics-tls" not found Jul 30 06:45:06.372969 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:45:06.372907 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert podName:52df3cd9-75cc-47c9-b997-48295db78d16 nodeName:}" failed. No retries permitted until 2026-07-30 06:45:38.372896052 +0000 UTC m=+99.729900191 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-svjsc" (UID: "52df3cd9-75cc-47c9-b997-48295db78d16") : secret "networking-console-plugin-cert" not found Jul 30 06:45:06.372969 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:45:06.372930 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 06:45:06.372969 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:45:06.372945 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77d84d85c6-wscbq: secret "image-registry-tls" not found Jul 30 06:45:06.373824 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:45:06.373006 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls podName:f1ba0bcb-7d3d-447e-8272-4910b15167d3 nodeName:}" failed. No retries permitted until 2026-07-30 06:45:38.372988841 +0000 UTC m=+99.729993017 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls") pod "image-registry-77d84d85c6-wscbq" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3") : secret "image-registry-tls" not found Jul 30 06:45:10.458829 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:10.458790 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sc2rl" event={"ID":"dd5d594e-9943-4d46-97e0-44cbbdc6ff6d","Type":"ContainerStarted","Data":"869b363513546f037fde097576b5ca375132955322e08f9eda455a38d17960aa"} Jul 30 06:45:10.460264 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:10.460237 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fr6kg" event={"ID":"b5281138-d6b6-4c65-a18f-6b153031868e","Type":"ContainerStarted","Data":"bf4f3a9ee2749635cfd4bb4c5d9aba89d68cc47854c836417d30f35f56b9afa7"} Jul 30 06:45:10.460393 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:10.460375 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:45:10.475283 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:10.475234 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sc2rl" podStartSLOduration=66.788455738 podStartE2EDuration="1m11.475222544s" totalImagesPullingTime="4.686766806s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 06:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:45:10.474927307 +0000 UTC m=+71.831931469" watchObservedRunningTime="2026-07-30 06:45:10.475222544 +0000 UTC m=+71.832226731" Jul 30 06:45:10.488820 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:10.488773 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fr6kg" podStartSLOduration=67.10684697 podStartE2EDuration="1m11.488759586s" totalImagesPullingTime="4.381912616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:45:10.48811774 +0000 UTC m=+71.845121901" watchObservedRunningTime="2026-07-30 06:45:10.488759586 +0000 UTC m=+71.845763748" Jul 30 06:45:38.399225 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:38.399184 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:45:38.399225 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:38.399226 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-svjsc\" (UID: \"52df3cd9-75cc-47c9-b997-48295db78d16\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:45:38.399756 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:45:38.399332 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 06:45:38.399756 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:45:38.399359 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77d84d85c6-wscbq: secret "image-registry-tls" not found Jul 30 06:45:38.399756 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:45:38.399381 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 06:45:38.399756 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:45:38.399416 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls podName:f1ba0bcb-7d3d-447e-8272-4910b15167d3 nodeName:}" failed. No retries permitted until 2026-07-30 06:46:42.399396544 +0000 UTC m=+163.756400684 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls") pod "image-registry-77d84d85c6-wscbq" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3") : secret "image-registry-tls" not found Jul 30 06:45:38.399756 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:45:38.399434 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert podName:52df3cd9-75cc-47c9-b997-48295db78d16 nodeName:}" failed. No retries permitted until 2026-07-30 06:46:42.399422296 +0000 UTC m=+163.756426440 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-svjsc" (UID: "52df3cd9-75cc-47c9-b997-48295db78d16") : secret "networking-console-plugin-cert" not found Jul 30 06:45:38.399756 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:38.399465 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:45:38.399756 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:45:38.399530 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 06:45:38.399756 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:45:38.399559 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls podName:e85b5491-8587-430a-ac24-a541065ee2da nodeName:}" failed. No retries permitted until 2026-07-30 06:46:42.399552143 +0000 UTC m=+163.756556285 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls") pod "dns-default-pvswz" (UID: "e85b5491-8587-430a-ac24-a541065ee2da") : secret "dns-default-metrics-tls" not found Jul 30 06:45:41.465565 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:45:41.465533 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fr6kg" Jul 30 06:46:09.017423 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:46:09.017374 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs\") pod \"network-metrics-daemon-7tcdf\" (UID: \"2a6fa524-6766-4ecd-a86a-5f26f29a72b2\") " pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:46:09.017923 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:46:09.017530 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 30 06:46:09.017923 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:46:09.017604 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs podName:2a6fa524-6766-4ecd-a86a-5f26f29a72b2 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:11.017586586 +0000 UTC m=+252.374590727 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs") pod "network-metrics-daemon-7tcdf" (UID: "2a6fa524-6766-4ecd-a86a-5f26f29a72b2") : secret "metrics-daemon-secret" not found Jul 30 06:46:37.525078 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:46:37.525029 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" podUID="f1ba0bcb-7d3d-447e-8272-4910b15167d3" Jul 30 06:46:37.532235 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:46:37.532202 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" podUID="52df3cd9-75cc-47c9-b997-48295db78d16" Jul 30 06:46:37.541519 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:46:37.541489 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-pvswz" podUID="e85b5491-8587-430a-ac24-a541065ee2da" Jul 30 06:46:37.666386 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:46:37.666303 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:46:37.666386 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:46:37.666350 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pvswz" Jul 30 06:46:37.666640 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:46:37.666620 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:46:38.188974 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:46:38.188927 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:46:42.455284 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:46:42.455232 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:46:42.455284 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:46:42.455289 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-svjsc\" (UID: \"52df3cd9-75cc-47c9-b997-48295db78d16\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:46:42.455863 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:46:42.455339 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:46:42.455863 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:46:42.455385 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 30 06:46:42.455863 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:46:42.455412 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77d84d85c6-wscbq: secret "image-registry-tls" not found Jul 30 06:46:42.455863 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:46:42.455468 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls podName:f1ba0bcb-7d3d-447e-8272-4910b15167d3 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:44.455452975 +0000 UTC m=+285.812457116 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls") pod "image-registry-77d84d85c6-wscbq" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3") : secret "image-registry-tls" not found Jul 30 06:46:42.455863 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:46:42.455469 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 30 06:46:42.455863 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:46:42.455469 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 30 06:46:42.455863 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:46:42.455521 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls podName:e85b5491-8587-430a-ac24-a541065ee2da nodeName:}" failed. No retries permitted until 2026-07-30 06:48:44.455506572 +0000 UTC m=+285.812510729 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls") pod "dns-default-pvswz" (UID: "e85b5491-8587-430a-ac24-a541065ee2da") : secret "dns-default-metrics-tls" not found Jul 30 06:46:42.455863 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:46:42.455537 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert podName:52df3cd9-75cc-47c9-b997-48295db78d16 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:44.45553007 +0000 UTC m=+285.812534220 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-svjsc" (UID: "52df3cd9-75cc-47c9-b997-48295db78d16") : secret "networking-console-plugin-cert" not found Jul 30 06:46:52.171122 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:46:52.171037 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:48:10.169869 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.169831 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm"] Jul 30 06:48:10.171848 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.171819 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:10.172343 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.172327 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76"] Jul 30 06:48:10.173926 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.173908 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" Jul 30 06:48:10.177733 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.177707 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vznhg\"" Jul 30 06:48:10.177989 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.177973 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 30 06:48:10.178215 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.178200 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 30 06:48:10.178745 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.178721 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 30 06:48:10.179016 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.178997 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 30 06:48:10.179553 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.179534 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 30 06:48:10.180244 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.180222 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6sllm\"" Jul 30 06:48:10.180371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.180345 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 30 06:48:10.180834 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.180816 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 30 06:48:10.181110 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.181092 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-794pk"] Jul 30 06:48:10.182996 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.182979 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-sflbp"] Jul 30 06:48:10.183208 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.183186 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:10.184928 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.184880 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm"] Jul 30 06:48:10.185032 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.185007 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.185888 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.185868 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 30 06:48:10.186344 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.185908 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 30 06:48:10.186344 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.185937 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 30 06:48:10.186681 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.186647 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76"] Jul 30 06:48:10.186838 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.186689 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 30 06:48:10.186952 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.186721 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-kmpsh\"" Jul 30 06:48:10.187982 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.187964 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 30 06:48:10.188154 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.188039 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 30 06:48:10.188242 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.188185 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-fvh67\"" Jul 30 06:48:10.188304 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.188102 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 30 06:48:10.188359 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.188080 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 30 06:48:10.190887 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.190867 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-794pk"] Jul 30 06:48:10.192760 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.192735 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 30 06:48:10.196143 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.196123 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 30 06:48:10.205326 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.205302 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-sflbp"] Jul 30 06:48:10.283027 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.282996 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pptkm\" (UID: \"b6eedc85-4962-46e8-a390-8e8547fbd0db\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:10.283027 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.283032 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0697e5d7-9959-4776-82b9-4a6f3e66b7c3-trusted-ca\") pod \"console-operator-584dcb6b54-794pk\" (UID: \"0697e5d7-9959-4776-82b9-4a6f3e66b7c3\") " pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:10.283228 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.283052 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0697e5d7-9959-4776-82b9-4a6f3e66b7c3-config\") pod \"console-operator-584dcb6b54-794pk\" (UID: \"0697e5d7-9959-4776-82b9-4a6f3e66b7c3\") " pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:10.283228 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.283105 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bdbbfc08-6f67-4fda-b22a-4c00d352d582-serving-cert\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.283228 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.283132 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0697e5d7-9959-4776-82b9-4a6f3e66b7c3-serving-cert\") pod \"console-operator-584dcb6b54-794pk\" (UID: \"0697e5d7-9959-4776-82b9-4a6f3e66b7c3\") " pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:10.283228 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.283185 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdbbfc08-6f67-4fda-b22a-4c00d352d582-service-ca-bundle\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.283228 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.283205 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjslz\" (UniqueName: \"kubernetes.io/projected/b6eedc85-4962-46e8-a390-8e8547fbd0db-kube-api-access-wjslz\") pod \"cluster-monitoring-operator-548b489cb8-pptkm\" (UID: \"b6eedc85-4962-46e8-a390-8e8547fbd0db\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:10.283430 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.283236 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b6eedc85-4962-46e8-a390-8e8547fbd0db-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-pptkm\" (UID: \"b6eedc85-4962-46e8-a390-8e8547fbd0db\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:10.283430 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.283420 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdbbfc08-6f67-4fda-b22a-4c00d352d582-tmp\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.283495 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.283450 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bdbbfc08-6f67-4fda-b22a-4c00d352d582-snapshots\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.283526 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.283486 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-q6c76\" (UID: \"232857dc-c3eb-408d-bcf1-e1b546443505\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" Jul 30 06:48:10.283558 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.283520 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5hcw\" (UniqueName: \"kubernetes.io/projected/0697e5d7-9959-4776-82b9-4a6f3e66b7c3-kube-api-access-j5hcw\") pod \"console-operator-584dcb6b54-794pk\" (UID: \"0697e5d7-9959-4776-82b9-4a6f3e66b7c3\") " pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:10.283558 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.283543 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdbbfc08-6f67-4fda-b22a-4c00d352d582-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.283620 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.283565 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-775kb\" (UniqueName: \"kubernetes.io/projected/232857dc-c3eb-408d-bcf1-e1b546443505-kube-api-access-775kb\") pod \"cluster-samples-operator-6748489656-q6c76\" (UID: \"232857dc-c3eb-408d-bcf1-e1b546443505\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" Jul 30 06:48:10.283620 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.283591 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf6pb\" (UniqueName: \"kubernetes.io/projected/bdbbfc08-6f67-4fda-b22a-4c00d352d582-kube-api-access-nf6pb\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.384565 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.384524 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bdbbfc08-6f67-4fda-b22a-4c00d352d582-serving-cert\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.384565 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.384563 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0697e5d7-9959-4776-82b9-4a6f3e66b7c3-serving-cert\") pod \"console-operator-584dcb6b54-794pk\" (UID: \"0697e5d7-9959-4776-82b9-4a6f3e66b7c3\") " pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:10.384857 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.384691 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdbbfc08-6f67-4fda-b22a-4c00d352d582-service-ca-bundle\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.384857 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.384725 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjslz\" (UniqueName: \"kubernetes.io/projected/b6eedc85-4962-46e8-a390-8e8547fbd0db-kube-api-access-wjslz\") pod \"cluster-monitoring-operator-548b489cb8-pptkm\" (UID: \"b6eedc85-4962-46e8-a390-8e8547fbd0db\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:10.384857 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.384756 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b6eedc85-4962-46e8-a390-8e8547fbd0db-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-pptkm\" (UID: \"b6eedc85-4962-46e8-a390-8e8547fbd0db\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:10.384857 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.384809 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdbbfc08-6f67-4fda-b22a-4c00d352d582-tmp\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.384857 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.384832 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bdbbfc08-6f67-4fda-b22a-4c00d352d582-snapshots\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.385107 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.384860 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-q6c76\" (UID: \"232857dc-c3eb-408d-bcf1-e1b546443505\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" Jul 30 06:48:10.385107 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.384889 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5hcw\" (UniqueName: \"kubernetes.io/projected/0697e5d7-9959-4776-82b9-4a6f3e66b7c3-kube-api-access-j5hcw\") pod \"console-operator-584dcb6b54-794pk\" (UID: \"0697e5d7-9959-4776-82b9-4a6f3e66b7c3\") " pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:10.385107 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.384913 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdbbfc08-6f67-4fda-b22a-4c00d352d582-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.385107 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.384938 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-775kb\" (UniqueName: \"kubernetes.io/projected/232857dc-c3eb-408d-bcf1-e1b546443505-kube-api-access-775kb\") pod \"cluster-samples-operator-6748489656-q6c76\" (UID: \"232857dc-c3eb-408d-bcf1-e1b546443505\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" Jul 30 06:48:10.385107 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.384975 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf6pb\" (UniqueName: \"kubernetes.io/projected/bdbbfc08-6f67-4fda-b22a-4c00d352d582-kube-api-access-nf6pb\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.385107 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.385031 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pptkm\" (UID: \"b6eedc85-4962-46e8-a390-8e8547fbd0db\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:10.385107 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.385063 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0697e5d7-9959-4776-82b9-4a6f3e66b7c3-trusted-ca\") pod \"console-operator-584dcb6b54-794pk\" (UID: \"0697e5d7-9959-4776-82b9-4a6f3e66b7c3\") " pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:10.385107 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.385092 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0697e5d7-9959-4776-82b9-4a6f3e66b7c3-config\") pod \"console-operator-584dcb6b54-794pk\" (UID: \"0697e5d7-9959-4776-82b9-4a6f3e66b7c3\") " pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:10.385673 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.385164 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdbbfc08-6f67-4fda-b22a-4c00d352d582-tmp\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.385673 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.385341 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdbbfc08-6f67-4fda-b22a-4c00d352d582-service-ca-bundle\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.385673 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:10.385427 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 06:48:10.385673 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:10.385479 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls podName:232857dc-c3eb-408d-bcf1-e1b546443505 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:10.885462827 +0000 UTC m=+252.242466968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls") pod "cluster-samples-operator-6748489656-q6c76" (UID: "232857dc-c3eb-408d-bcf1-e1b546443505") : secret "samples-operator-tls" not found Jul 30 06:48:10.385915 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:10.385767 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 06:48:10.385915 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:10.385847 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls podName:b6eedc85-4962-46e8-a390-8e8547fbd0db nodeName:}" failed. No retries permitted until 2026-07-30 06:48:10.88582696 +0000 UTC m=+252.242831109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-pptkm" (UID: "b6eedc85-4962-46e8-a390-8e8547fbd0db") : secret "cluster-monitoring-operator-tls" not found Jul 30 06:48:10.385915 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.385887 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdbbfc08-6f67-4fda-b22a-4c00d352d582-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.385915 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.385903 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b6eedc85-4962-46e8-a390-8e8547fbd0db-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-pptkm\" (UID: \"b6eedc85-4962-46e8-a390-8e8547fbd0db\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:10.386118 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.386039 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0697e5d7-9959-4776-82b9-4a6f3e66b7c3-config\") pod \"console-operator-584dcb6b54-794pk\" (UID: \"0697e5d7-9959-4776-82b9-4a6f3e66b7c3\") " pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:10.386181 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.386155 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bdbbfc08-6f67-4fda-b22a-4c00d352d582-snapshots\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.387064 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.387039 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0697e5d7-9959-4776-82b9-4a6f3e66b7c3-trusted-ca\") pod \"console-operator-584dcb6b54-794pk\" (UID: \"0697e5d7-9959-4776-82b9-4a6f3e66b7c3\") " pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:10.387289 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.387271 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0697e5d7-9959-4776-82b9-4a6f3e66b7c3-serving-cert\") pod \"console-operator-584dcb6b54-794pk\" (UID: \"0697e5d7-9959-4776-82b9-4a6f3e66b7c3\") " pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:10.387363 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.387349 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bdbbfc08-6f67-4fda-b22a-4c00d352d582-serving-cert\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.392726 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.392706 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjslz\" (UniqueName: \"kubernetes.io/projected/b6eedc85-4962-46e8-a390-8e8547fbd0db-kube-api-access-wjslz\") pod \"cluster-monitoring-operator-548b489cb8-pptkm\" (UID: \"b6eedc85-4962-46e8-a390-8e8547fbd0db\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:10.395962 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.395943 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-775kb\" (UniqueName: \"kubernetes.io/projected/232857dc-c3eb-408d-bcf1-e1b546443505-kube-api-access-775kb\") pod \"cluster-samples-operator-6748489656-q6c76\" (UID: \"232857dc-c3eb-408d-bcf1-e1b546443505\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" Jul 30 06:48:10.396733 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.396714 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5hcw\" (UniqueName: \"kubernetes.io/projected/0697e5d7-9959-4776-82b9-4a6f3e66b7c3-kube-api-access-j5hcw\") pod \"console-operator-584dcb6b54-794pk\" (UID: \"0697e5d7-9959-4776-82b9-4a6f3e66b7c3\") " pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:10.396970 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.396948 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf6pb\" (UniqueName: \"kubernetes.io/projected/bdbbfc08-6f67-4fda-b22a-4c00d352d582-kube-api-access-nf6pb\") pod \"insights-operator-5885fdb759-sflbp\" (UID: \"bdbbfc08-6f67-4fda-b22a-4c00d352d582\") " pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.502770 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.502687 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:10.508266 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.508246 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-sflbp" Jul 30 06:48:10.631149 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.631119 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-794pk"] Jul 30 06:48:10.635286 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:48:10.635260 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0697e5d7_9959_4776_82b9_4a6f3e66b7c3.slice/crio-5888b02a77ee29afd3952453646ace60485874e87b50c2d06edd816325b3132b WatchSource:0}: Error finding container 5888b02a77ee29afd3952453646ace60485874e87b50c2d06edd816325b3132b: Status 404 returned error can't find the container with id 5888b02a77ee29afd3952453646ace60485874e87b50c2d06edd816325b3132b Jul 30 06:48:10.648728 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.648703 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-sflbp"] Jul 30 06:48:10.651431 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:48:10.651406 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdbbfc08_6f67_4fda_b22a_4c00d352d582.slice/crio-02ab5d1a55f73d58a8295b36ea087a3c5065fe0dc1c65a6274f3c0e5f67d0f73 WatchSource:0}: Error finding container 02ab5d1a55f73d58a8295b36ea087a3c5065fe0dc1c65a6274f3c0e5f67d0f73: Status 404 returned error can't find the container with id 02ab5d1a55f73d58a8295b36ea087a3c5065fe0dc1c65a6274f3c0e5f67d0f73 Jul 30 06:48:10.885824 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.885788 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-sflbp" event={"ID":"bdbbfc08-6f67-4fda-b22a-4c00d352d582","Type":"ContainerStarted","Data":"02ab5d1a55f73d58a8295b36ea087a3c5065fe0dc1c65a6274f3c0e5f67d0f73"} Jul 30 06:48:10.886834 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.886811 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-794pk" event={"ID":"0697e5d7-9959-4776-82b9-4a6f3e66b7c3","Type":"ContainerStarted","Data":"5888b02a77ee29afd3952453646ace60485874e87b50c2d06edd816325b3132b"} Jul 30 06:48:10.888145 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.888129 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-q6c76\" (UID: \"232857dc-c3eb-408d-bcf1-e1b546443505\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" Jul 30 06:48:10.888194 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:10.888177 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pptkm\" (UID: \"b6eedc85-4962-46e8-a390-8e8547fbd0db\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:10.888274 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:10.888262 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 06:48:10.888321 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:10.888274 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 06:48:10.888321 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:10.888308 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls podName:b6eedc85-4962-46e8-a390-8e8547fbd0db nodeName:}" failed. No retries permitted until 2026-07-30 06:48:11.888295341 +0000 UTC m=+253.245299481 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-pptkm" (UID: "b6eedc85-4962-46e8-a390-8e8547fbd0db") : secret "cluster-monitoring-operator-tls" not found Jul 30 06:48:10.888410 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:10.888335 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls podName:232857dc-c3eb-408d-bcf1-e1b546443505 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:11.888322249 +0000 UTC m=+253.245326389 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls") pod "cluster-samples-operator-6748489656-q6c76" (UID: "232857dc-c3eb-408d-bcf1-e1b546443505") : secret "samples-operator-tls" not found Jul 30 06:48:11.089529 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:11.089476 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs\") pod \"network-metrics-daemon-7tcdf\" (UID: \"2a6fa524-6766-4ecd-a86a-5f26f29a72b2\") " pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:48:11.089738 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:11.089626 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 30 06:48:11.089738 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:11.089706 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs podName:2a6fa524-6766-4ecd-a86a-5f26f29a72b2 nodeName:}" failed. No retries permitted until 2026-07-30 06:50:13.089692134 +0000 UTC m=+374.446696278 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs") pod "network-metrics-daemon-7tcdf" (UID: "2a6fa524-6766-4ecd-a86a-5f26f29a72b2") : secret "metrics-daemon-secret" not found Jul 30 06:48:11.896452 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:11.896417 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-q6c76\" (UID: \"232857dc-c3eb-408d-bcf1-e1b546443505\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" Jul 30 06:48:11.896965 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:11.896500 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pptkm\" (UID: \"b6eedc85-4962-46e8-a390-8e8547fbd0db\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:11.896965 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:11.896606 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 06:48:11.896965 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:11.896627 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 06:48:11.896965 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:11.896700 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls podName:b6eedc85-4962-46e8-a390-8e8547fbd0db nodeName:}" failed. No retries permitted until 2026-07-30 06:48:13.896681952 +0000 UTC m=+255.253686095 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-pptkm" (UID: "b6eedc85-4962-46e8-a390-8e8547fbd0db") : secret "cluster-monitoring-operator-tls" not found Jul 30 06:48:11.896965 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:11.896719 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls podName:232857dc-c3eb-408d-bcf1-e1b546443505 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:13.896709124 +0000 UTC m=+255.253713268 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls") pod "cluster-samples-operator-6748489656-q6c76" (UID: "232857dc-c3eb-408d-bcf1-e1b546443505") : secret "samples-operator-tls" not found Jul 30 06:48:13.897436 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:13.897411 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/0.log" Jul 30 06:48:13.897870 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:13.897451 2575 generic.go:356] "Generic (PLEG): container finished" podID="0697e5d7-9959-4776-82b9-4a6f3e66b7c3" containerID="4a2897e8e3f7cb9c827f75845e4f323dc433b50dea1a0a7a7bce590a41b8bdb8" exitCode=255 Jul 30 06:48:13.897870 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:13.897528 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-794pk" event={"ID":"0697e5d7-9959-4776-82b9-4a6f3e66b7c3","Type":"ContainerDied","Data":"4a2897e8e3f7cb9c827f75845e4f323dc433b50dea1a0a7a7bce590a41b8bdb8"} Jul 30 06:48:13.897870 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:13.897789 2575 scope.go:117] "RemoveContainer" containerID="4a2897e8e3f7cb9c827f75845e4f323dc433b50dea1a0a7a7bce590a41b8bdb8" Jul 30 06:48:13.898859 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:13.898836 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-sflbp" event={"ID":"bdbbfc08-6f67-4fda-b22a-4c00d352d582","Type":"ContainerStarted","Data":"a732760366ad111faf857eb0cefad8ae54370075b313df88dc8953280a2b1c51"} Jul 30 06:48:13.913403 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:13.913376 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pptkm\" (UID: \"b6eedc85-4962-46e8-a390-8e8547fbd0db\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:13.913553 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:13.913534 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 06:48:13.913621 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:13.913599 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls podName:b6eedc85-4962-46e8-a390-8e8547fbd0db nodeName:}" failed. No retries permitted until 2026-07-30 06:48:17.913580921 +0000 UTC m=+259.270585078 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-pptkm" (UID: "b6eedc85-4962-46e8-a390-8e8547fbd0db") : secret "cluster-monitoring-operator-tls" not found Jul 30 06:48:13.913743 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:13.913625 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 06:48:13.913743 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:13.913701 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls podName:232857dc-c3eb-408d-bcf1-e1b546443505 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:17.913683907 +0000 UTC m=+259.270688054 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls") pod "cluster-samples-operator-6748489656-q6c76" (UID: "232857dc-c3eb-408d-bcf1-e1b546443505") : secret "samples-operator-tls" not found Jul 30 06:48:13.913918 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:13.913544 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-q6c76\" (UID: \"232857dc-c3eb-408d-bcf1-e1b546443505\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" Jul 30 06:48:13.928080 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:13.928039 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-sflbp" podStartSLOduration=1.100732082 podStartE2EDuration="3.928023821s" totalImagesPullingTime="2.827291739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:48:13.926568869 +0000 UTC m=+255.283573031" watchObservedRunningTime="2026-07-30 06:48:13.928023821 +0000 UTC m=+255.285027984" Jul 30 06:48:14.089202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.089164 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7jxg6"] Jul 30 06:48:14.091532 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.091508 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:14.094567 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.094541 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lpmmm\"" Jul 30 06:48:14.094739 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.094700 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 30 06:48:14.094885 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.094864 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 30 06:48:14.110030 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.110004 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7jxg6"] Jul 30 06:48:14.115134 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.115114 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e95e6dbb-61a0-4d2b-8600-591817523c32-crio-socket\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:14.115251 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.115139 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:14.115251 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.115180 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:14.115251 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.115223 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lh48\" (UniqueName: \"kubernetes.io/projected/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-api-access-5lh48\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:14.115423 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.115302 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e95e6dbb-61a0-4d2b-8600-591817523c32-data-volume\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:14.215886 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.215800 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lh48\" (UniqueName: \"kubernetes.io/projected/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-api-access-5lh48\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:14.215886 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.215838 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e95e6dbb-61a0-4d2b-8600-591817523c32-data-volume\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:14.216074 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.215927 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e95e6dbb-61a0-4d2b-8600-591817523c32-crio-socket\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:14.216074 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.215966 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:14.216074 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.215985 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e95e6dbb-61a0-4d2b-8600-591817523c32-crio-socket\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:14.216074 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.216040 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:14.216074 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:14.216062 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 06:48:14.216258 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:14.216090 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 06:48:14.216258 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:14.216121 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls podName:e95e6dbb-61a0-4d2b-8600-591817523c32 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:14.716101881 +0000 UTC m=+256.073106020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7jxg6" (UID: "e95e6dbb-61a0-4d2b-8600-591817523c32") : secret "insights-runtime-extractor-tls" not found Jul 30 06:48:14.216258 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:14.216141 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm podName:e95e6dbb-61a0-4d2b-8600-591817523c32 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:14.716127128 +0000 UTC m=+256.073131273 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7jxg6" (UID: "e95e6dbb-61a0-4d2b-8600-591817523c32") : configmap "kube-rbac-proxy" not found Jul 30 06:48:14.216361 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.216322 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e95e6dbb-61a0-4d2b-8600-591817523c32-data-volume\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:14.224410 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.224378 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lh48\" (UniqueName: \"kubernetes.io/projected/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-api-access-5lh48\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:14.701669 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.701632 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-l9vxw"] Jul 30 06:48:14.703966 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.703948 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-l9vxw" Jul 30 06:48:14.706637 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.706613 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 30 06:48:14.706768 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.706672 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 30 06:48:14.706768 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.706701 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-g7jd4\"" Jul 30 06:48:14.712473 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.712451 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-l9vxw"] Jul 30 06:48:14.719301 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.719278 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkgc2\" (UniqueName: \"kubernetes.io/projected/c167d2e1-1e5f-439a-9cc9-aed32eb9316a-kube-api-access-rkgc2\") pod \"migrator-6865c9967-l9vxw\" (UID: \"c167d2e1-1e5f-439a-9cc9-aed32eb9316a\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-l9vxw" Jul 30 06:48:14.719389 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.719322 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:14.719389 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.719365 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:14.719502 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:14.719449 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 06:48:14.719502 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:14.719473 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 06:48:14.719559 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:14.719508 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm podName:e95e6dbb-61a0-4d2b-8600-591817523c32 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:15.719490657 +0000 UTC m=+257.076494817 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7jxg6" (UID: "e95e6dbb-61a0-4d2b-8600-591817523c32") : configmap "kube-rbac-proxy" not found Jul 30 06:48:14.719559 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:14.719522 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls podName:e95e6dbb-61a0-4d2b-8600-591817523c32 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:15.719516481 +0000 UTC m=+257.076520621 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7jxg6" (UID: "e95e6dbb-61a0-4d2b-8600-591817523c32") : secret "insights-runtime-extractor-tls" not found Jul 30 06:48:14.820613 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.820574 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkgc2\" (UniqueName: \"kubernetes.io/projected/c167d2e1-1e5f-439a-9cc9-aed32eb9316a-kube-api-access-rkgc2\") pod \"migrator-6865c9967-l9vxw\" (UID: \"c167d2e1-1e5f-439a-9cc9-aed32eb9316a\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-l9vxw" Jul 30 06:48:14.828156 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.828135 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkgc2\" (UniqueName: \"kubernetes.io/projected/c167d2e1-1e5f-439a-9cc9-aed32eb9316a-kube-api-access-rkgc2\") pod \"migrator-6865c9967-l9vxw\" (UID: \"c167d2e1-1e5f-439a-9cc9-aed32eb9316a\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-l9vxw" Jul 30 06:48:14.906448 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.906414 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/1.log" Jul 30 06:48:14.907068 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.906999 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/0.log" Jul 30 06:48:14.907068 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.907032 2575 generic.go:356] "Generic (PLEG): container finished" podID="0697e5d7-9959-4776-82b9-4a6f3e66b7c3" containerID="9d663de37e5a75132bdc873646b90f3eab6ae7fe6b16beae5320c4bafefb7a96" exitCode=255 Jul 30 06:48:14.907185 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.907127 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-794pk" event={"ID":"0697e5d7-9959-4776-82b9-4a6f3e66b7c3","Type":"ContainerDied","Data":"9d663de37e5a75132bdc873646b90f3eab6ae7fe6b16beae5320c4bafefb7a96"} Jul 30 06:48:14.907340 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.907315 2575 scope.go:117] "RemoveContainer" containerID="4a2897e8e3f7cb9c827f75845e4f323dc433b50dea1a0a7a7bce590a41b8bdb8" Jul 30 06:48:14.907847 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:14.907827 2575 scope.go:117] "RemoveContainer" containerID="9d663de37e5a75132bdc873646b90f3eab6ae7fe6b16beae5320c4bafefb7a96" Jul 30 06:48:14.908048 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:14.908018 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-794pk_openshift-console-operator(0697e5d7-9959-4776-82b9-4a6f3e66b7c3)\"" pod="openshift-console-operator/console-operator-584dcb6b54-794pk" podUID="0697e5d7-9959-4776-82b9-4a6f3e66b7c3" Jul 30 06:48:15.013493 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:15.013419 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-l9vxw" Jul 30 06:48:15.135301 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:15.135269 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-l9vxw"] Jul 30 06:48:15.137520 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:48:15.137496 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc167d2e1_1e5f_439a_9cc9_aed32eb9316a.slice/crio-56910be2cbb544f02e1cafb4f73b852fe98455a2e24651b226b3c0a8d70877d5 WatchSource:0}: Error finding container 56910be2cbb544f02e1cafb4f73b852fe98455a2e24651b226b3c0a8d70877d5: Status 404 returned error can't find the container with id 56910be2cbb544f02e1cafb4f73b852fe98455a2e24651b226b3c0a8d70877d5 Jul 30 06:48:15.728109 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:15.728075 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:15.728347 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:15.728136 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:15.728347 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:15.728215 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 06:48:15.728347 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:15.728217 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 06:48:15.728347 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:15.728273 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm podName:e95e6dbb-61a0-4d2b-8600-591817523c32 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:17.72825528 +0000 UTC m=+259.085259421 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7jxg6" (UID: "e95e6dbb-61a0-4d2b-8600-591817523c32") : configmap "kube-rbac-proxy" not found Jul 30 06:48:15.728347 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:15.728287 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls podName:e95e6dbb-61a0-4d2b-8600-591817523c32 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:17.728281335 +0000 UTC m=+259.085285475 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7jxg6" (UID: "e95e6dbb-61a0-4d2b-8600-591817523c32") : secret "insights-runtime-extractor-tls" not found Jul 30 06:48:15.911414 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:15.911372 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/1.log" Jul 30 06:48:15.911882 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:15.911783 2575 scope.go:117] "RemoveContainer" containerID="9d663de37e5a75132bdc873646b90f3eab6ae7fe6b16beae5320c4bafefb7a96" Jul 30 06:48:15.912051 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:15.912022 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-794pk_openshift-console-operator(0697e5d7-9959-4776-82b9-4a6f3e66b7c3)\"" pod="openshift-console-operator/console-operator-584dcb6b54-794pk" podUID="0697e5d7-9959-4776-82b9-4a6f3e66b7c3" Jul 30 06:48:15.912619 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:15.912593 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-l9vxw" event={"ID":"c167d2e1-1e5f-439a-9cc9-aed32eb9316a","Type":"ContainerStarted","Data":"56910be2cbb544f02e1cafb4f73b852fe98455a2e24651b226b3c0a8d70877d5"} Jul 30 06:48:15.981944 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:15.981858 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7p8zb_2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9/dns-node-resolver/0.log" Jul 30 06:48:16.917487 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:16.917447 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-l9vxw" event={"ID":"c167d2e1-1e5f-439a-9cc9-aed32eb9316a","Type":"ContainerStarted","Data":"bbdbdb26b2da09401a6e46d02eb1a4f7c45885da1b5575b99a0d7e80fb5bc9ad"} Jul 30 06:48:16.917873 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:16.917508 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-l9vxw" event={"ID":"c167d2e1-1e5f-439a-9cc9-aed32eb9316a","Type":"ContainerStarted","Data":"96f99920807716a2cb978abc946752b36218ff40d21deefc85270a92f94ec22d"} Jul 30 06:48:16.932707 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:16.932641 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-l9vxw" podStartSLOduration=1.547610797 podStartE2EDuration="2.932625846s" totalImagesPullingTime="1.385015049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:48:16.931692514 +0000 UTC m=+258.288696670" watchObservedRunningTime="2026-07-30 06:48:16.932625846 +0000 UTC m=+258.289630008" Jul 30 06:48:17.381959 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:17.381930 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xmcq2_7adaf671-8982-4075-92cd-d190635a2fe7/node-ca/0.log" Jul 30 06:48:17.746887 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:17.746806 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:17.746887 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:17.746862 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:17.747069 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:17.746957 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 06:48:17.747069 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:17.746971 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 06:48:17.747069 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:17.747021 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls podName:e95e6dbb-61a0-4d2b-8600-591817523c32 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:21.747006141 +0000 UTC m=+263.104010286 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7jxg6" (UID: "e95e6dbb-61a0-4d2b-8600-591817523c32") : secret "insights-runtime-extractor-tls" not found Jul 30 06:48:17.747069 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:17.747036 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm podName:e95e6dbb-61a0-4d2b-8600-591817523c32 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:21.747030501 +0000 UTC m=+263.104034644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7jxg6" (UID: "e95e6dbb-61a0-4d2b-8600-591817523c32") : configmap "kube-rbac-proxy" not found Jul 30 06:48:17.948098 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:17.948068 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-q6c76\" (UID: \"232857dc-c3eb-408d-bcf1-e1b546443505\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" Jul 30 06:48:17.948471 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:17.948145 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pptkm\" (UID: \"b6eedc85-4962-46e8-a390-8e8547fbd0db\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:17.948471 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:17.948211 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 30 06:48:17.948471 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:17.948245 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 06:48:17.948471 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:17.948280 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls podName:232857dc-c3eb-408d-bcf1-e1b546443505 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:25.948262226 +0000 UTC m=+267.305266384 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls") pod "cluster-samples-operator-6748489656-q6c76" (UID: "232857dc-c3eb-408d-bcf1-e1b546443505") : secret "samples-operator-tls" not found Jul 30 06:48:17.948471 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:17.948293 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls podName:b6eedc85-4962-46e8-a390-8e8547fbd0db nodeName:}" failed. No retries permitted until 2026-07-30 06:48:25.948287684 +0000 UTC m=+267.305291824 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-pptkm" (UID: "b6eedc85-4962-46e8-a390-8e8547fbd0db") : secret "cluster-monitoring-operator-tls" not found Jul 30 06:48:17.981554 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:17.981528 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bnkvv_510fa8de-adc5-476f-bb43-50233a5f291b/serve-healthcheck-canary/0.log" Jul 30 06:48:20.503849 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:20.503755 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:20.503849 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:20.503853 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:20.504232 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:20.504172 2575 scope.go:117] "RemoveContainer" containerID="9d663de37e5a75132bdc873646b90f3eab6ae7fe6b16beae5320c4bafefb7a96" Jul 30 06:48:20.504324 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:20.504304 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-794pk_openshift-console-operator(0697e5d7-9959-4776-82b9-4a6f3e66b7c3)\"" pod="openshift-console-operator/console-operator-584dcb6b54-794pk" podUID="0697e5d7-9959-4776-82b9-4a6f3e66b7c3" Jul 30 06:48:21.781858 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:21.781823 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:21.782230 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:21.781877 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:21.782230 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:21.781964 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 30 06:48:21.782230 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:21.781965 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 30 06:48:21.782230 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:21.782024 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls podName:e95e6dbb-61a0-4d2b-8600-591817523c32 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:29.782009352 +0000 UTC m=+271.139013496 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7jxg6" (UID: "e95e6dbb-61a0-4d2b-8600-591817523c32") : secret "insights-runtime-extractor-tls" not found Jul 30 06:48:21.782230 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:21.782039 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm podName:e95e6dbb-61a0-4d2b-8600-591817523c32 nodeName:}" failed. No retries permitted until 2026-07-30 06:48:29.782032273 +0000 UTC m=+271.139036413 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7jxg6" (UID: "e95e6dbb-61a0-4d2b-8600-591817523c32") : configmap "kube-rbac-proxy" not found Jul 30 06:48:26.014514 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:26.014456 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-q6c76\" (UID: \"232857dc-c3eb-408d-bcf1-e1b546443505\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" Jul 30 06:48:26.015026 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:26.014549 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pptkm\" (UID: \"b6eedc85-4962-46e8-a390-8e8547fbd0db\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:26.015026 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:26.014692 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 30 06:48:26.015026 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:26.014751 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls podName:b6eedc85-4962-46e8-a390-8e8547fbd0db nodeName:}" failed. No retries permitted until 2026-07-30 06:48:42.014736785 +0000 UTC m=+283.371740924 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-pptkm" (UID: "b6eedc85-4962-46e8-a390-8e8547fbd0db") : secret "cluster-monitoring-operator-tls" not found Jul 30 06:48:26.017079 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:26.017053 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/232857dc-c3eb-408d-bcf1-e1b546443505-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-q6c76\" (UID: \"232857dc-c3eb-408d-bcf1-e1b546443505\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" Jul 30 06:48:26.096882 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:26.096854 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" Jul 30 06:48:26.219494 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:26.219463 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76"] Jul 30 06:48:26.948411 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:26.948370 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" event={"ID":"232857dc-c3eb-408d-bcf1-e1b546443505","Type":"ContainerStarted","Data":"5c28827d89b43799423cefb5750650d7b05d1636728f474e1fe1acd132b2ffe9"} Jul 30 06:48:28.955201 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:28.955166 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" event={"ID":"232857dc-c3eb-408d-bcf1-e1b546443505","Type":"ContainerStarted","Data":"a640f538e3f62a7f12f5d8701f1601fd8d518d3e6ce17bc5a9d4ddb08c1d068b"} Jul 30 06:48:28.955615 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:28.955220 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" event={"ID":"232857dc-c3eb-408d-bcf1-e1b546443505","Type":"ContainerStarted","Data":"2d91e672a5835dd9f798be4437de62c6d943ecf99fb5c0ea28ddea5df37b0171"} Jul 30 06:48:28.972141 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:28.972097 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-q6c76" podStartSLOduration=17.333875018 podStartE2EDuration="18.972083119s" totalImagesPullingTime="1.638208101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:48:28.970708802 +0000 UTC m=+270.327712964" watchObservedRunningTime="2026-07-30 06:48:28.972083119 +0000 UTC m=+270.329087281" Jul 30 06:48:29.848085 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:29.848047 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:29.848265 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:29.848107 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:29.848711 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:29.848689 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e95e6dbb-61a0-4d2b-8600-591817523c32-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:29.850516 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:29.850492 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e95e6dbb-61a0-4d2b-8600-591817523c32-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7jxg6\" (UID: \"e95e6dbb-61a0-4d2b-8600-591817523c32\") " pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:30.003954 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:30.003915 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7jxg6" Jul 30 06:48:30.128074 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:30.128048 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7jxg6"] Jul 30 06:48:30.131495 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:48:30.131467 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode95e6dbb_61a0_4d2b_8600_591817523c32.slice/crio-74f4cc3e9a2e349f2ffdafd34d5605e7a14a3c3f1792c2d0d9a74ca7112ceaca WatchSource:0}: Error finding container 74f4cc3e9a2e349f2ffdafd34d5605e7a14a3c3f1792c2d0d9a74ca7112ceaca: Status 404 returned error can't find the container with id 74f4cc3e9a2e349f2ffdafd34d5605e7a14a3c3f1792c2d0d9a74ca7112ceaca Jul 30 06:48:30.964045 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:30.963952 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7jxg6" event={"ID":"e95e6dbb-61a0-4d2b-8600-591817523c32","Type":"ContainerStarted","Data":"74f4cc3e9a2e349f2ffdafd34d5605e7a14a3c3f1792c2d0d9a74ca7112ceaca"} Jul 30 06:48:33.171758 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:33.171725 2575 scope.go:117] "RemoveContainer" containerID="9d663de37e5a75132bdc873646b90f3eab6ae7fe6b16beae5320c4bafefb7a96" Jul 30 06:48:35.984096 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:35.983977 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7jxg6" event={"ID":"e95e6dbb-61a0-4d2b-8600-591817523c32","Type":"ContainerStarted","Data":"8866efb339f6374ab6b99babf910a529a70d73e6036795880fd2dd15a8ad2109"} Jul 30 06:48:35.986939 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:35.986910 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/1.log" Jul 30 06:48:35.987722 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:35.987368 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-794pk" event={"ID":"0697e5d7-9959-4776-82b9-4a6f3e66b7c3","Type":"ContainerStarted","Data":"f9a3420c27e0c58735cdf151f14e65fffffe236d3fe5bc7c791713211005df23"} Jul 30 06:48:35.992180 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:35.988207 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:36.005063 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:36.005012 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-794pk" podStartSLOduration=23.16314416 podStartE2EDuration="26.004992293s" totalImagesPullingTime="2.841848133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:48:36.003678299 +0000 UTC m=+277.360682462" watchObservedRunningTime="2026-07-30 06:48:36.004992293 +0000 UTC m=+277.361996456" Jul 30 06:48:36.444124 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:36.444093 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-794pk" Jul 30 06:48:37.619643 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:37.619611 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-gfn77"] Jul 30 06:48:37.635648 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:37.635616 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-gfn77" Jul 30 06:48:37.636284 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:37.636261 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-gfn77"] Jul 30 06:48:37.638817 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:37.638359 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 30 06:48:37.638817 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:37.638362 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-gr5pl\"" Jul 30 06:48:37.638817 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:37.638408 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 30 06:48:37.716301 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:37.716269 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpb9z\" (UniqueName: \"kubernetes.io/projected/52145120-8982-4e8f-9b50-dcfa914cdb41-kube-api-access-cpb9z\") pod \"downloads-7c97dc4555-gfn77\" (UID: \"52145120-8982-4e8f-9b50-dcfa914cdb41\") " pod="openshift-console/downloads-7c97dc4555-gfn77" Jul 30 06:48:37.816917 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:37.816881 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpb9z\" (UniqueName: \"kubernetes.io/projected/52145120-8982-4e8f-9b50-dcfa914cdb41-kube-api-access-cpb9z\") pod \"downloads-7c97dc4555-gfn77\" (UID: \"52145120-8982-4e8f-9b50-dcfa914cdb41\") " pod="openshift-console/downloads-7c97dc4555-gfn77" Jul 30 06:48:37.825557 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:37.825528 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpb9z\" (UniqueName: \"kubernetes.io/projected/52145120-8982-4e8f-9b50-dcfa914cdb41-kube-api-access-cpb9z\") pod \"downloads-7c97dc4555-gfn77\" (UID: \"52145120-8982-4e8f-9b50-dcfa914cdb41\") " pod="openshift-console/downloads-7c97dc4555-gfn77" Jul 30 06:48:37.949857 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:37.949824 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-gfn77" Jul 30 06:48:38.988142 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:38.988118 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-gfn77"] Jul 30 06:48:38.998680 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:38.998639 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7jxg6" event={"ID":"e95e6dbb-61a0-4d2b-8600-591817523c32","Type":"ContainerStarted","Data":"863bd41f438173998791e5cff7535b62fa37beaf953080685f77ef1d605cec3f"} Jul 30 06:48:38.999649 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:38.999626 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-gfn77" event={"ID":"52145120-8982-4e8f-9b50-dcfa914cdb41","Type":"ContainerStarted","Data":"98c3a6f047bd7cc9627af3c0bd74eb751a9a43556a38910ae6073bec9f181b34"} Jul 30 06:48:40.667591 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:40.667512 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" podUID="f1ba0bcb-7d3d-447e-8272-4910b15167d3" Jul 30 06:48:40.667591 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:40.667512 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" podUID="52df3cd9-75cc-47c9-b997-48295db78d16" Jul 30 06:48:40.667591 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:40.667526 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-pvswz" podUID="e85b5491-8587-430a-ac24-a541065ee2da" Jul 30 06:48:41.008729 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:41.008482 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pvswz" Jul 30 06:48:41.008729 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:41.008517 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:48:41.008729 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:41.008517 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:48:42.057958 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:42.057918 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pptkm\" (UID: \"b6eedc85-4962-46e8-a390-8e8547fbd0db\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:42.060919 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:42.060870 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b6eedc85-4962-46e8-a390-8e8547fbd0db-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-pptkm\" (UID: \"b6eedc85-4962-46e8-a390-8e8547fbd0db\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:42.288092 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:42.288061 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" Jul 30 06:48:43.175445 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:43.175421 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm"] Jul 30 06:48:43.176035 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:48:43.176006 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6eedc85_4962_46e8_a390_8e8547fbd0db.slice/crio-b8a8e34e2cf4a92588afda01b80df9c4c37c44dc55f4eb3f048184ccc967a986 WatchSource:0}: Error finding container b8a8e34e2cf4a92588afda01b80df9c4c37c44dc55f4eb3f048184ccc967a986: Status 404 returned error can't find the container with id b8a8e34e2cf4a92588afda01b80df9c4c37c44dc55f4eb3f048184ccc967a986 Jul 30 06:48:44.023743 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:44.023706 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7jxg6" event={"ID":"e95e6dbb-61a0-4d2b-8600-591817523c32","Type":"ContainerStarted","Data":"36cb647a48ffe5778d96267ace64f2cf3f984dfcb4c88d2f980edcbc03247cbe"} Jul 30 06:48:44.025223 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:44.025181 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" event={"ID":"b6eedc85-4962-46e8-a390-8e8547fbd0db","Type":"ContainerStarted","Data":"b8a8e34e2cf4a92588afda01b80df9c4c37c44dc55f4eb3f048184ccc967a986"} Jul 30 06:48:44.482475 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:44.482440 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:48:44.482974 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:44.482496 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:48:44.482974 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:44.482577 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-svjsc\" (UID: \"52df3cd9-75cc-47c9-b997-48295db78d16\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:48:44.485772 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:44.485730 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e85b5491-8587-430a-ac24-a541065ee2da-metrics-tls\") pod \"dns-default-pvswz\" (UID: \"e85b5491-8587-430a-ac24-a541065ee2da\") " pod="openshift-dns/dns-default-pvswz" Jul 30 06:48:44.485772 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:44.485749 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls\") pod \"image-registry-77d84d85c6-wscbq\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:48:44.486312 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:44.486288 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/52df3cd9-75cc-47c9-b997-48295db78d16-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-svjsc\" (UID: \"52df3cd9-75cc-47c9-b997-48295db78d16\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:48:44.612759 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:44.612712 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cvd6x\"" Jul 30 06:48:44.612938 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:44.612786 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f2g8l\"" Jul 30 06:48:44.612938 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:44.612787 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-mwlhp\"" Jul 30 06:48:44.620178 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:44.620152 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" Jul 30 06:48:44.620346 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:44.620196 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pvswz" Jul 30 06:48:44.620346 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:44.620313 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:48:45.363375 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:45.362977 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7jxg6" podStartSLOduration=18.536035024 podStartE2EDuration="31.362961175s" totalImagesPullingTime="12.826926151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 06:48:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:48:44.041574568 +0000 UTC m=+285.398578731" watchObservedRunningTime="2026-07-30 06:48:45.362961175 +0000 UTC m=+286.719965337" Jul 30 06:48:45.364017 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:45.363989 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-svjsc"] Jul 30 06:48:45.369835 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:48:45.369805 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52df3cd9_75cc_47c9_b997_48295db78d16.slice/crio-abd5417af59744eecbf38f00b74ac629ebc77bdc6bb02e5d2d6e7d4d83cfafeb WatchSource:0}: Error finding container abd5417af59744eecbf38f00b74ac629ebc77bdc6bb02e5d2d6e7d4d83cfafeb: Status 404 returned error can't find the container with id abd5417af59744eecbf38f00b74ac629ebc77bdc6bb02e5d2d6e7d4d83cfafeb Jul 30 06:48:45.392052 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:45.392031 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pvswz"] Jul 30 06:48:45.393354 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:45.393306 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77d84d85c6-wscbq"] Jul 30 06:48:45.394381 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:48:45.394323 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode85b5491_8587_430a_ac24_a541065ee2da.slice/crio-12cf6ef5b01fe948f866fc20b8c5282c6213abe5ceaf844097a9141b1b336274 WatchSource:0}: Error finding container 12cf6ef5b01fe948f866fc20b8c5282c6213abe5ceaf844097a9141b1b336274: Status 404 returned error can't find the container with id 12cf6ef5b01fe948f866fc20b8c5282c6213abe5ceaf844097a9141b1b336274 Jul 30 06:48:45.397008 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:48:45.396979 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1ba0bcb_7d3d_447e_8272_4910b15167d3.slice/crio-1cf9b330120f574834775af415653a7bc6ca6d4b44c60135edfacd93f97a2fe2 WatchSource:0}: Error finding container 1cf9b330120f574834775af415653a7bc6ca6d4b44c60135edfacd93f97a2fe2: Status 404 returned error can't find the container with id 1cf9b330120f574834775af415653a7bc6ca6d4b44c60135edfacd93f97a2fe2 Jul 30 06:48:46.034675 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.034625 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" event={"ID":"b6eedc85-4962-46e8-a390-8e8547fbd0db","Type":"ContainerStarted","Data":"afec64cfbb28be0dc607f1b6b0de6b41d77cb0b8b178cb44f811be1575fa1fb0"} Jul 30 06:48:46.036460 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.036397 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" event={"ID":"52df3cd9-75cc-47c9-b997-48295db78d16","Type":"ContainerStarted","Data":"abd5417af59744eecbf38f00b74ac629ebc77bdc6bb02e5d2d6e7d4d83cfafeb"} Jul 30 06:48:46.038468 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.038433 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" event={"ID":"f1ba0bcb-7d3d-447e-8272-4910b15167d3","Type":"ContainerStarted","Data":"864c0a38e179893e8df109af747a3e011498a662d38e2ebc0211a470bc0540dc"} Jul 30 06:48:46.038644 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.038476 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" event={"ID":"f1ba0bcb-7d3d-447e-8272-4910b15167d3","Type":"ContainerStarted","Data":"1cf9b330120f574834775af415653a7bc6ca6d4b44c60135edfacd93f97a2fe2"} Jul 30 06:48:46.038644 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.038502 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:48:46.044605 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.040167 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pvswz" event={"ID":"e85b5491-8587-430a-ac24-a541065ee2da","Type":"ContainerStarted","Data":"12cf6ef5b01fe948f866fc20b8c5282c6213abe5ceaf844097a9141b1b336274"} Jul 30 06:48:46.054605 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.054551 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-pptkm" podStartSLOduration=34.007294519 podStartE2EDuration="36.054533174s" totalImagesPullingTime="2.047238655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:48:46.05313348 +0000 UTC m=+287.410137642" watchObservedRunningTime="2026-07-30 06:48:46.054533174 +0000 UTC m=+287.411537338" Jul 30 06:48:46.076433 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.076388 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" podStartSLOduration=287.076371852 podStartE2EDuration="4m47.076371852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:43:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:48:46.075865608 +0000 UTC m=+287.432869772" watchObservedRunningTime="2026-07-30 06:48:46.076371852 +0000 UTC m=+287.433376017" Jul 30 06:48:46.742961 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.741767 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85cdd5d96b-gc469"] Jul 30 06:48:46.774480 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.774445 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85cdd5d96b-gc469"] Jul 30 06:48:46.774637 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.774564 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:46.777738 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.777701 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 30 06:48:46.778765 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.778739 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 30 06:48:46.778886 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.778830 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 30 06:48:46.778886 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.778845 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 30 06:48:46.778886 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.778845 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wb5d4\"" Jul 30 06:48:46.779060 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.778844 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 30 06:48:46.905393 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.905355 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0f863f9-00cc-49fd-886a-518e3c977931-console-serving-cert\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:46.905574 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.905408 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-service-ca\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:46.905574 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.905465 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-console-config\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:46.905574 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.905487 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-oauth-serving-cert\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:46.905789 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.905610 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0f863f9-00cc-49fd-886a-518e3c977931-console-oauth-config\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:46.905789 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:46.905650 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh5xp\" (UniqueName: \"kubernetes.io/projected/e0f863f9-00cc-49fd-886a-518e3c977931-kube-api-access-zh5xp\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:47.006957 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:47.006878 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0f863f9-00cc-49fd-886a-518e3c977931-console-serving-cert\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:47.006957 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:47.006923 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-service-ca\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:47.007183 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:47.007055 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-console-config\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:47.007183 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:47.007095 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-oauth-serving-cert\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:47.007183 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:47.007176 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0f863f9-00cc-49fd-886a-518e3c977931-console-oauth-config\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:47.007348 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:47.007201 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh5xp\" (UniqueName: \"kubernetes.io/projected/e0f863f9-00cc-49fd-886a-518e3c977931-kube-api-access-zh5xp\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:47.007711 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:47.007682 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-service-ca\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:47.007866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:47.007755 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-console-config\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:47.007866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:47.007810 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-oauth-serving-cert\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:47.009880 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:47.009855 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0f863f9-00cc-49fd-886a-518e3c977931-console-oauth-config\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:47.009992 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:47.009926 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0f863f9-00cc-49fd-886a-518e3c977931-console-serving-cert\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:47.017500 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:47.017473 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh5xp\" (UniqueName: \"kubernetes.io/projected/e0f863f9-00cc-49fd-886a-518e3c977931-kube-api-access-zh5xp\") pod \"console-85cdd5d96b-gc469\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:47.086825 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:47.086788 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:48:47.856611 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:47.856541 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85cdd5d96b-gc469"] Jul 30 06:48:47.860565 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:48:47.860530 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0f863f9_00cc_49fd_886a_518e3c977931.slice/crio-22449c68d1c8c77b34c90eb428bf1d585d1e3e30b4ddf62fdd35432bb59b436b WatchSource:0}: Error finding container 22449c68d1c8c77b34c90eb428bf1d585d1e3e30b4ddf62fdd35432bb59b436b: Status 404 returned error can't find the container with id 22449c68d1c8c77b34c90eb428bf1d585d1e3e30b4ddf62fdd35432bb59b436b Jul 30 06:48:48.051466 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.051377 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pvswz" event={"ID":"e85b5491-8587-430a-ac24-a541065ee2da","Type":"ContainerStarted","Data":"4380da2cf79286dc553cdb6acb316a3bd33157aa9c3f59a190a29e1224d892fb"} Jul 30 06:48:48.051466 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.051441 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pvswz" event={"ID":"e85b5491-8587-430a-ac24-a541065ee2da","Type":"ContainerStarted","Data":"6772042ea96a6e425a34affac5eb1bdbad4f3105b212777c46d96b93aa7693b2"} Jul 30 06:48:48.051642 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.051491 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pvswz" Jul 30 06:48:48.053155 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.053127 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cdd5d96b-gc469" event={"ID":"e0f863f9-00cc-49fd-886a-518e3c977931","Type":"ContainerStarted","Data":"22449c68d1c8c77b34c90eb428bf1d585d1e3e30b4ddf62fdd35432bb59b436b"} Jul 30 06:48:48.054673 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.054624 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" event={"ID":"52df3cd9-75cc-47c9-b997-48295db78d16","Type":"ContainerStarted","Data":"cbe5746aaa152bf497eddede8e3812337bfc83065de31b15783d95f4af81c875"} Jul 30 06:48:48.072300 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.072253 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pvswz" podStartSLOduration=251.758959147 podStartE2EDuration="4m14.072234581s" totalImagesPullingTime="2.313275434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:44:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:48:48.07082396 +0000 UTC m=+289.427828122" watchObservedRunningTime="2026-07-30 06:48:48.072234581 +0000 UTC m=+289.429238743" Jul 30 06:48:48.089069 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.089017 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-svjsc" podStartSLOduration=278.759147295 podStartE2EDuration="4m41.088986471s" totalImagesPullingTime="2.329839176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:44:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:48:48.088569536 +0000 UTC m=+289.445573699" watchObservedRunningTime="2026-07-30 06:48:48.088986471 +0000 UTC m=+289.445990650" Jul 30 06:48:48.837532 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.837497 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-7v92k"] Jul 30 06:48:48.840975 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.840953 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" Jul 30 06:48:48.844517 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.844483 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 30 06:48:48.844517 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.844493 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 30 06:48:48.844736 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.844491 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 30 06:48:48.844736 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.844492 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hhsx9\"" Jul 30 06:48:48.849267 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.848929 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-7v92k"] Jul 30 06:48:48.922222 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.922106 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f24f984f-a413-416a-a941-876b3531376e-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-7v92k\" (UID: \"f24f984f-a413-416a-a941-876b3531376e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" Jul 30 06:48:48.922222 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.922163 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9wk9\" (UniqueName: \"kubernetes.io/projected/f24f984f-a413-416a-a941-876b3531376e-kube-api-access-p9wk9\") pod \"prometheus-operator-84d69f794d-7v92k\" (UID: \"f24f984f-a413-416a-a941-876b3531376e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" Jul 30 06:48:48.922222 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.922194 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f24f984f-a413-416a-a941-876b3531376e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-7v92k\" (UID: \"f24f984f-a413-416a-a941-876b3531376e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" Jul 30 06:48:48.922497 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:48.922240 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f24f984f-a413-416a-a941-876b3531376e-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-7v92k\" (UID: \"f24f984f-a413-416a-a941-876b3531376e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" Jul 30 06:48:49.023339 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:49.023300 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f24f984f-a413-416a-a941-876b3531376e-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-7v92k\" (UID: \"f24f984f-a413-416a-a941-876b3531376e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" Jul 30 06:48:49.023536 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:49.023364 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9wk9\" (UniqueName: \"kubernetes.io/projected/f24f984f-a413-416a-a941-876b3531376e-kube-api-access-p9wk9\") pod \"prometheus-operator-84d69f794d-7v92k\" (UID: \"f24f984f-a413-416a-a941-876b3531376e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" Jul 30 06:48:49.023536 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:49.023405 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f24f984f-a413-416a-a941-876b3531376e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-7v92k\" (UID: \"f24f984f-a413-416a-a941-876b3531376e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" Jul 30 06:48:49.023536 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:49.023437 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f24f984f-a413-416a-a941-876b3531376e-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-7v92k\" (UID: \"f24f984f-a413-416a-a941-876b3531376e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" Jul 30 06:48:49.024276 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:49.024246 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f24f984f-a413-416a-a941-876b3531376e-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-7v92k\" (UID: \"f24f984f-a413-416a-a941-876b3531376e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" Jul 30 06:48:49.026927 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:49.026902 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f24f984f-a413-416a-a941-876b3531376e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-7v92k\" (UID: \"f24f984f-a413-416a-a941-876b3531376e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" Jul 30 06:48:49.027322 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:49.027297 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f24f984f-a413-416a-a941-876b3531376e-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-7v92k\" (UID: \"f24f984f-a413-416a-a941-876b3531376e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" Jul 30 06:48:49.033282 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:49.033245 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9wk9\" (UniqueName: \"kubernetes.io/projected/f24f984f-a413-416a-a941-876b3531376e-kube-api-access-p9wk9\") pod \"prometheus-operator-84d69f794d-7v92k\" (UID: \"f24f984f-a413-416a-a941-876b3531376e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" Jul 30 06:48:49.152399 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:49.152348 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" Jul 30 06:48:49.314278 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:49.314244 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-7v92k"] Jul 30 06:48:55.172048 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:48:55.171998 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-7tcdf" podUID="2a6fa524-6766-4ecd-a86a-5f26f29a72b2" Jul 30 06:48:56.498575 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:48:56.498544 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf24f984f_a413_416a_a941_876b3531376e.slice/crio-d03bc0c48a2aebe8fdb92f551b452165d50e4b3890bf5caa4393cf32929ddf44 WatchSource:0}: Error finding container d03bc0c48a2aebe8fdb92f551b452165d50e4b3890bf5caa4393cf32929ddf44: Status 404 returned error can't find the container with id d03bc0c48a2aebe8fdb92f551b452165d50e4b3890bf5caa4393cf32929ddf44 Jul 30 06:48:57.092155 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:57.092114 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" event={"ID":"f24f984f-a413-416a-a941-876b3531376e","Type":"ContainerStarted","Data":"d03bc0c48a2aebe8fdb92f551b452165d50e4b3890bf5caa4393cf32929ddf44"} Jul 30 06:48:58.061360 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:58.061339 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pvswz" Jul 30 06:48:58.097783 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:58.097624 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cdd5d96b-gc469" event={"ID":"e0f863f9-00cc-49fd-886a-518e3c977931","Type":"ContainerStarted","Data":"8c8153ce57495c3d2da999b92dbbda83c9a6e5229b3a960b9fab4a885aa8d793"} Jul 30 06:48:58.099238 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:58.099198 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-gfn77" event={"ID":"52145120-8982-4e8f-9b50-dcfa914cdb41","Type":"ContainerStarted","Data":"f62abcad9feac6278be4e1cb0a06becfb93ff41550e585a017ba6c666af62740"} Jul 30 06:48:58.099353 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:58.099256 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-gfn77" Jul 30 06:48:58.100615 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:58.100586 2575 patch_prober.go:28] interesting pod/downloads-7c97dc4555-gfn77 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.16:8080/\": dial tcp 10.132.0.16:8080: connect: connection refused" start-of-body= Jul 30 06:48:58.100736 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:58.100642 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-gfn77" podUID="52145120-8982-4e8f-9b50-dcfa914cdb41" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.16:8080/\": dial tcp 10.132.0.16:8080: connect: connection refused" Jul 30 06:48:58.119330 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:58.119248 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85cdd5d96b-gc469" podStartSLOduration=2.003862384 podStartE2EDuration="12.119231167s" totalImagesPullingTime="10.115368783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:48:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:48:58.118544034 +0000 UTC m=+299.475548199" watchObservedRunningTime="2026-07-30 06:48:58.119231167 +0000 UTC m=+299.476235307" Jul 30 06:48:58.135858 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:58.135804 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-gfn77" podStartSLOduration=2.112562053 podStartE2EDuration="21.135785319s" totalImagesPullingTime="19.023223266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:48:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:48:58.13417494 +0000 UTC m=+299.491179101" watchObservedRunningTime="2026-07-30 06:48:58.135785319 +0000 UTC m=+299.492789480" Jul 30 06:48:59.115643 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:59.115616 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-gfn77" Jul 30 06:48:59.201902 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:59.201871 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/1.log" Jul 30 06:48:59.954716 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:59.202480 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/1.log" Jul 30 06:48:59.954716 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:59.212609 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 06:48:59.954716 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:59.212648 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 06:48:59.954716 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:48:59.216334 2575 kubelet.go:1648] "Image garbage collection succeeded" Jul 30 06:49:00.111831 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:00.111789 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" event={"ID":"f24f984f-a413-416a-a941-876b3531376e","Type":"ContainerStarted","Data":"f03c7cdf155599e09bf5f2089ddb23b93de0e62c15681b6010b139c16725baf5"} Jul 30 06:49:00.112015 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:00.111851 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" event={"ID":"f24f984f-a413-416a-a941-876b3531376e","Type":"ContainerStarted","Data":"ef4646770ff078d1eb62bc8e894c6b7a578ed4b837f15fb7d360388b0aef1551"} Jul 30 06:49:00.128867 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:00.128813 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-7v92k" podStartSLOduration=9.118008344 podStartE2EDuration="12.128793296s" totalImagesPullingTime="3.010784952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:48:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:49:00.126788419 +0000 UTC m=+301.483792605" watchObservedRunningTime="2026-07-30 06:49:00.128793296 +0000 UTC m=+301.485797459" Jul 30 06:49:01.383482 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:01.383445 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77d84d85c6-wscbq"] Jul 30 06:49:01.388847 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:01.388818 2575 patch_prober.go:28] interesting pod/image-registry-77d84d85c6-wscbq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 30 06:49:01.388999 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:01.388874 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" podUID="f1ba0bcb-7d3d-447e-8272-4910b15167d3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 30 06:49:02.180717 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.180677 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gglzd"] Jul 30 06:49:02.226046 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.226016 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.228771 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.228743 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 30 06:49:02.228913 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.228798 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lm9kk\"" Jul 30 06:49:02.228983 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.228925 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 30 06:49:02.229208 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.229188 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 30 06:49:02.246824 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.246803 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-node-exporter-textfile\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.246935 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.246835 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-root\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.246935 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.246882 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-node-exporter-wtmp\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.246935 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.246917 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-node-exporter-accelerators-collector-config\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.247101 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.246944 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-sys\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.247101 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.246971 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.247101 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.247001 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-metrics-client-ca\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.247101 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.247066 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-node-exporter-tls\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.247267 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.247114 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtk2g\" (UniqueName: \"kubernetes.io/projected/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-kube-api-access-jtk2g\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.348060 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.348022 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-root\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.348242 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.348093 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-node-exporter-wtmp\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.348242 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.348131 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-root\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.348242 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.348131 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-node-exporter-accelerators-collector-config\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.348242 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.348191 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-sys\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.348242 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.348220 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.348527 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.348246 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-metrics-client-ca\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.348527 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.348286 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-node-exporter-tls\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.348527 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.348286 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-node-exporter-wtmp\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.348527 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.348315 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-sys\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.348527 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.348352 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtk2g\" (UniqueName: \"kubernetes.io/projected/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-kube-api-access-jtk2g\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.348527 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.348412 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-node-exporter-textfile\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.348893 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.348723 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-node-exporter-textfile\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.348893 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.348735 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-node-exporter-accelerators-collector-config\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.348893 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.348795 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-metrics-client-ca\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.351132 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.351097 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.351341 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.351322 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-node-exporter-tls\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.356559 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.356516 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtk2g\" (UniqueName: \"kubernetes.io/projected/d05d0a5d-8ebf-4a60-88d4-69bb32d2f440-kube-api-access-jtk2g\") pod \"node-exporter-gglzd\" (UID: \"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440\") " pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.537078 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.536996 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gglzd" Jul 30 06:49:02.582992 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:49:02.582950 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd05d0a5d_8ebf_4a60_88d4_69bb32d2f440.slice/crio-94c232f23a6113df9948fc6ea597145b5418188fc00f5d50ca4eedf20b208a18 WatchSource:0}: Error finding container 94c232f23a6113df9948fc6ea597145b5418188fc00f5d50ca4eedf20b208a18: Status 404 returned error can't find the container with id 94c232f23a6113df9948fc6ea597145b5418188fc00f5d50ca4eedf20b208a18 Jul 30 06:49:02.585244 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:02.585223 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 06:49:03.126367 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.126327 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gglzd" event={"ID":"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440","Type":"ContainerStarted","Data":"94c232f23a6113df9948fc6ea597145b5418188fc00f5d50ca4eedf20b208a18"} Jul 30 06:49:03.237235 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.237196 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 06:49:03.254982 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.254899 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 06:49:03.255156 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.255048 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.258073 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.258049 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 30 06:49:03.258285 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.258266 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 30 06:49:03.259065 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.258524 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 30 06:49:03.259303 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.258618 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 30 06:49:03.259561 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.258685 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 30 06:49:03.259704 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.258765 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 30 06:49:03.260030 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.258806 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jhc44\"" Jul 30 06:49:03.260030 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.258840 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 30 06:49:03.260030 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.258874 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 30 06:49:03.260030 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.258901 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 30 06:49:03.357866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.357309 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/96d17342-cda6-4016-b1fb-8f3378febafc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.357866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.357370 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.357866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.357418 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96d17342-cda6-4016-b1fb-8f3378febafc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.357866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.357453 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.357866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.357496 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.357866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.357583 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-config-volume\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.357866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.357618 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/96d17342-cda6-4016-b1fb-8f3378febafc-config-out\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.357866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.357677 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-web-config\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.357866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.357713 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.357866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.357740 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvfl8\" (UniqueName: \"kubernetes.io/projected/96d17342-cda6-4016-b1fb-8f3378febafc-kube-api-access-lvfl8\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.357866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.357773 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/96d17342-cda6-4016-b1fb-8f3378febafc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.357866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.357800 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.357866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.357839 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96d17342-cda6-4016-b1fb-8f3378febafc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.459693 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.459188 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.459693 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.459268 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96d17342-cda6-4016-b1fb-8f3378febafc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.459693 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.459297 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.459693 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.459339 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.459693 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.459382 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-config-volume\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.459693 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.459413 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/96d17342-cda6-4016-b1fb-8f3378febafc-config-out\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.459693 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.459444 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-web-config\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.459693 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.459470 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.459693 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.459485 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvfl8\" (UniqueName: \"kubernetes.io/projected/96d17342-cda6-4016-b1fb-8f3378febafc-kube-api-access-lvfl8\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.459693 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.459507 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/96d17342-cda6-4016-b1fb-8f3378febafc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.459693 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.459522 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.459693 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.459561 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96d17342-cda6-4016-b1fb-8f3378febafc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.459693 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.459593 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/96d17342-cda6-4016-b1fb-8f3378febafc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.461186 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.461152 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96d17342-cda6-4016-b1fb-8f3378febafc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.461316 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:49:03.461266 2575 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 30 06:49:03.461377 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:49:03.461322 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-main-tls podName:96d17342-cda6-4016-b1fb-8f3378febafc nodeName:}" failed. No retries permitted until 2026-07-30 06:49:03.961302812 +0000 UTC m=+305.318306954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "96d17342-cda6-4016-b1fb-8f3378febafc") : secret "alertmanager-main-tls" not found Jul 30 06:49:03.462300 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.461526 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/96d17342-cda6-4016-b1fb-8f3378febafc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.463904 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.463850 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96d17342-cda6-4016-b1fb-8f3378febafc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.463904 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.463883 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-config-volume\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.465246 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.465223 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/96d17342-cda6-4016-b1fb-8f3378febafc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.465389 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.465372 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.467217 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.467175 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-web-config\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.467588 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.467545 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/96d17342-cda6-4016-b1fb-8f3378febafc-config-out\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.468206 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.467828 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.468312 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.468280 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.468778 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.468730 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvfl8\" (UniqueName: \"kubernetes.io/projected/96d17342-cda6-4016-b1fb-8f3378febafc-kube-api-access-lvfl8\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.470331 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.470307 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.965670 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.965617 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:03.968849 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:03.968814 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:04.170049 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:04.170005 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:49:04.558514 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:04.558472 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 06:49:04.560706 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:49:04.560674 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96d17342_cda6_4016_b1fb_8f3378febafc.slice/crio-d44b841b2b076500f27733796ffc2f0cc238364a584157b1d74dc8014d8c3a7d WatchSource:0}: Error finding container d44b841b2b076500f27733796ffc2f0cc238364a584157b1d74dc8014d8c3a7d: Status 404 returned error can't find the container with id d44b841b2b076500f27733796ffc2f0cc238364a584157b1d74dc8014d8c3a7d Jul 30 06:49:05.138212 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:05.138165 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96d17342-cda6-4016-b1fb-8f3378febafc","Type":"ContainerStarted","Data":"d44b841b2b076500f27733796ffc2f0cc238364a584157b1d74dc8014d8c3a7d"} Jul 30 06:49:05.141831 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:05.141064 2575 generic.go:356] "Generic (PLEG): container finished" podID="d05d0a5d-8ebf-4a60-88d4-69bb32d2f440" containerID="7a7f0e1f239739ccf6ab1c12bcc9311148c7c54dd142833bba1b87a0bc75e3fe" exitCode=0 Jul 30 06:49:05.141831 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:05.141105 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gglzd" event={"ID":"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440","Type":"ContainerDied","Data":"7a7f0e1f239739ccf6ab1c12bcc9311148c7c54dd142833bba1b87a0bc75e3fe"} Jul 30 06:49:06.147968 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:06.147931 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gglzd" event={"ID":"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440","Type":"ContainerStarted","Data":"5fc1e86043e8041b40324ad6d34e0d2df6239f62c64e3f679d68406a6cd47d84"} Jul 30 06:49:06.148432 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:06.147993 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gglzd" event={"ID":"d05d0a5d-8ebf-4a60-88d4-69bb32d2f440","Type":"ContainerStarted","Data":"6163250eb2c36ace5eb77708dbafaa4f01c7f249ef1fe04a3a4f6decbae41e29"} Jul 30 06:49:06.167736 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:06.167673 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gglzd" podStartSLOduration=2.252952456 podStartE2EDuration="4.167637793s" totalImagesPullingTime="1.812942432s" totalInitContainerRuntime="101.742905ms" isStatefulPod=true podCreationTimestamp="2026-07-30 06:49:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:49:06.165492855 +0000 UTC m=+307.522497011" watchObservedRunningTime="2026-07-30 06:49:06.167637793 +0000 UTC m=+307.524641954" Jul 30 06:49:06.503425 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:06.499525 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85cdd5d96b-gc469"] Jul 30 06:49:07.086924 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:07.086891 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:49:07.154324 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:07.154293 2575 generic.go:356] "Generic (PLEG): container finished" podID="96d17342-cda6-4016-b1fb-8f3378febafc" containerID="b84ac8fa11a7c21ab84eb0e18d45c429c044aae36538cc22e02ce31edd45d784" exitCode=0 Jul 30 06:49:07.154802 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:07.154387 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96d17342-cda6-4016-b1fb-8f3378febafc","Type":"ContainerDied","Data":"b84ac8fa11a7c21ab84eb0e18d45c429c044aae36538cc22e02ce31edd45d784"} Jul 30 06:49:08.171674 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:08.171623 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:49:09.145939 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.145897 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cc4596cdf-vt55p"] Jul 30 06:49:09.173229 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.173206 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cc4596cdf-vt55p"] Jul 30 06:49:09.173587 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.173332 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.181341 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.181314 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 30 06:49:09.219869 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.219830 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-trusted-ca-bundle\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.220021 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.219902 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3e757c4-bb98-457a-9992-5b738f8a8249-console-serving-cert\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.220021 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.219939 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkkn6\" (UniqueName: \"kubernetes.io/projected/b3e757c4-bb98-457a-9992-5b738f8a8249-kube-api-access-pkkn6\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.220021 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.219988 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3e757c4-bb98-457a-9992-5b738f8a8249-console-oauth-config\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.220125 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.220028 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-console-config\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.220125 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.220077 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-oauth-serving-cert\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.220187 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.220141 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-service-ca\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.320916 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.320885 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-service-ca\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.321084 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.320928 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-trusted-ca-bundle\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.321084 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.320948 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3e757c4-bb98-457a-9992-5b738f8a8249-console-serving-cert\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.321084 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.320969 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkkn6\" (UniqueName: \"kubernetes.io/projected/b3e757c4-bb98-457a-9992-5b738f8a8249-kube-api-access-pkkn6\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.321084 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.320994 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3e757c4-bb98-457a-9992-5b738f8a8249-console-oauth-config\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.321296 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.321180 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-console-config\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.321296 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.321221 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-oauth-serving-cert\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.321725 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.321697 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-service-ca\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.321855 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.321792 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-trusted-ca-bundle\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.321855 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.321841 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-console-config\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.321954 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.321915 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-oauth-serving-cert\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.324015 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.323989 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3e757c4-bb98-457a-9992-5b738f8a8249-console-serving-cert\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.324130 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.324093 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3e757c4-bb98-457a-9992-5b738f8a8249-console-oauth-config\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.328949 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.328927 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkkn6\" (UniqueName: \"kubernetes.io/projected/b3e757c4-bb98-457a-9992-5b738f8a8249-kube-api-access-pkkn6\") pod \"console-7cc4596cdf-vt55p\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.483303 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.483277 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:09.636056 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:09.636034 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cc4596cdf-vt55p"] Jul 30 06:49:09.638212 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:49:09.638178 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3e757c4_bb98_457a_9992_5b738f8a8249.slice/crio-576114c7059613d7a5fb3bdcc11f73af151c60df6f8bc8755e054c7cf9227299 WatchSource:0}: Error finding container 576114c7059613d7a5fb3bdcc11f73af151c60df6f8bc8755e054c7cf9227299: Status 404 returned error can't find the container with id 576114c7059613d7a5fb3bdcc11f73af151c60df6f8bc8755e054c7cf9227299 Jul 30 06:49:10.170580 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:10.170538 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96d17342-cda6-4016-b1fb-8f3378febafc","Type":"ContainerStarted","Data":"01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f"} Jul 30 06:49:10.170743 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:10.170678 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96d17342-cda6-4016-b1fb-8f3378febafc","Type":"ContainerStarted","Data":"015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5"} Jul 30 06:49:10.170743 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:10.170711 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96d17342-cda6-4016-b1fb-8f3378febafc","Type":"ContainerStarted","Data":"f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20"} Jul 30 06:49:10.170743 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:10.170742 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96d17342-cda6-4016-b1fb-8f3378febafc","Type":"ContainerStarted","Data":"cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b"} Jul 30 06:49:10.170913 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:10.170774 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96d17342-cda6-4016-b1fb-8f3378febafc","Type":"ContainerStarted","Data":"09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376"} Jul 30 06:49:10.172153 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:10.172128 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc4596cdf-vt55p" event={"ID":"b3e757c4-bb98-457a-9992-5b738f8a8249","Type":"ContainerStarted","Data":"5e9defcdd924427f571a0518b7cc3b2244c0a246fb6914931e5ea3bb561eac8b"} Jul 30 06:49:10.172242 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:10.172172 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc4596cdf-vt55p" event={"ID":"b3e757c4-bb98-457a-9992-5b738f8a8249","Type":"ContainerStarted","Data":"576114c7059613d7a5fb3bdcc11f73af151c60df6f8bc8755e054c7cf9227299"} Jul 30 06:49:10.189324 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:10.189280 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cc4596cdf-vt55p" podStartSLOduration=1.189264525 podStartE2EDuration="1.189264525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:49:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:49:10.187796929 +0000 UTC m=+311.544801092" watchObservedRunningTime="2026-07-30 06:49:10.189264525 +0000 UTC m=+311.546268687" Jul 30 06:49:11.388261 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:11.388189 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:49:12.184072 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:12.184035 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96d17342-cda6-4016-b1fb-8f3378febafc","Type":"ContainerStarted","Data":"b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20"} Jul 30 06:49:12.210814 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:12.210749 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.195342539 podStartE2EDuration="9.210726566s" totalImagesPullingTime="5.001759558s" totalInitContainerRuntime="13.624469ms" isStatefulPod=false podCreationTimestamp="2026-07-30 06:49:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:49:12.208751608 +0000 UTC m=+313.565755771" watchObservedRunningTime="2026-07-30 06:49:12.210726566 +0000 UTC m=+313.567730730" Jul 30 06:49:19.483610 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:19.483574 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:19.484077 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:19.483719 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:19.489717 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:19.489696 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:20.224558 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:20.224532 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:49:26.405272 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.405209 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" podUID="f1ba0bcb-7d3d-447e-8272-4910b15167d3" containerName="registry" containerID="cri-o://864c0a38e179893e8df109af747a3e011498a662d38e2ebc0211a470bc0540dc" gracePeriod=30 Jul 30 06:49:26.688281 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.688214 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:49:26.780730 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.780692 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7fjf\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-kube-api-access-f7fjf\") pod \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " Jul 30 06:49:26.780936 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.780744 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1ba0bcb-7d3d-447e-8272-4910b15167d3-image-registry-private-configuration\") pod \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " Jul 30 06:49:26.780936 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.780775 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls\") pod \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " Jul 30 06:49:26.780936 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.780814 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1ba0bcb-7d3d-447e-8272-4910b15167d3-installation-pull-secrets\") pod \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " Jul 30 06:49:26.780936 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.780845 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1ba0bcb-7d3d-447e-8272-4910b15167d3-ca-trust-extracted\") pod \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " Jul 30 06:49:26.780936 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.780868 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-certificates\") pod \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " Jul 30 06:49:26.781206 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.781105 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1ba0bcb-7d3d-447e-8272-4910b15167d3-trusted-ca\") pod \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " Jul 30 06:49:26.781206 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.781189 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-bound-sa-token\") pod \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\" (UID: \"f1ba0bcb-7d3d-447e-8272-4910b15167d3\") " Jul 30 06:49:26.781546 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.781387 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f1ba0bcb-7d3d-447e-8272-4910b15167d3" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:49:26.781546 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.781511 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1ba0bcb-7d3d-447e-8272-4910b15167d3-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f1ba0bcb-7d3d-447e-8272-4910b15167d3" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:49:26.781546 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.781519 2575 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-certificates\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:49:26.783372 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.783338 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f1ba0bcb-7d3d-447e-8272-4910b15167d3" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 06:49:26.783781 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.783749 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-kube-api-access-f7fjf" (OuterVolumeSpecName: "kube-api-access-f7fjf") pod "f1ba0bcb-7d3d-447e-8272-4910b15167d3" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3"). InnerVolumeSpecName "kube-api-access-f7fjf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 06:49:26.783781 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.783754 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1ba0bcb-7d3d-447e-8272-4910b15167d3-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f1ba0bcb-7d3d-447e-8272-4910b15167d3" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:49:26.783938 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.783886 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f1ba0bcb-7d3d-447e-8272-4910b15167d3" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 06:49:26.783938 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.783917 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1ba0bcb-7d3d-447e-8272-4910b15167d3-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f1ba0bcb-7d3d-447e-8272-4910b15167d3" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:49:26.789610 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.789584 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1ba0bcb-7d3d-447e-8272-4910b15167d3-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f1ba0bcb-7d3d-447e-8272-4910b15167d3" (UID: "f1ba0bcb-7d3d-447e-8272-4910b15167d3"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 06:49:26.881982 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.881948 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7fjf\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-kube-api-access-f7fjf\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:49:26.881982 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.881979 2575 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f1ba0bcb-7d3d-447e-8272-4910b15167d3-image-registry-private-configuration\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:49:26.882154 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.881991 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-registry-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:49:26.882154 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.881999 2575 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f1ba0bcb-7d3d-447e-8272-4910b15167d3-installation-pull-secrets\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:49:26.882154 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.882008 2575 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f1ba0bcb-7d3d-447e-8272-4910b15167d3-ca-trust-extracted\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:49:26.882154 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.882017 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f1ba0bcb-7d3d-447e-8272-4910b15167d3-trusted-ca\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:49:26.882154 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:26.882025 2575 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1ba0bcb-7d3d-447e-8272-4910b15167d3-bound-sa-token\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:49:27.247622 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:27.247590 2575 generic.go:356] "Generic (PLEG): container finished" podID="f1ba0bcb-7d3d-447e-8272-4910b15167d3" containerID="864c0a38e179893e8df109af747a3e011498a662d38e2ebc0211a470bc0540dc" exitCode=0 Jul 30 06:49:27.247794 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:27.247638 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" Jul 30 06:49:27.247794 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:27.247687 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" event={"ID":"f1ba0bcb-7d3d-447e-8272-4910b15167d3","Type":"ContainerDied","Data":"864c0a38e179893e8df109af747a3e011498a662d38e2ebc0211a470bc0540dc"} Jul 30 06:49:27.247794 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:27.247739 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77d84d85c6-wscbq" event={"ID":"f1ba0bcb-7d3d-447e-8272-4910b15167d3","Type":"ContainerDied","Data":"1cf9b330120f574834775af415653a7bc6ca6d4b44c60135edfacd93f97a2fe2"} Jul 30 06:49:27.247794 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:27.247759 2575 scope.go:117] "RemoveContainer" containerID="864c0a38e179893e8df109af747a3e011498a662d38e2ebc0211a470bc0540dc" Jul 30 06:49:27.256099 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:27.256082 2575 scope.go:117] "RemoveContainer" containerID="864c0a38e179893e8df109af747a3e011498a662d38e2ebc0211a470bc0540dc" Jul 30 06:49:27.256369 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:49:27.256348 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"864c0a38e179893e8df109af747a3e011498a662d38e2ebc0211a470bc0540dc\": container with ID starting with 864c0a38e179893e8df109af747a3e011498a662d38e2ebc0211a470bc0540dc not found: ID does not exist" containerID="864c0a38e179893e8df109af747a3e011498a662d38e2ebc0211a470bc0540dc" Jul 30 06:49:27.256418 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:27.256378 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"864c0a38e179893e8df109af747a3e011498a662d38e2ebc0211a470bc0540dc"} err="failed to get container status \"864c0a38e179893e8df109af747a3e011498a662d38e2ebc0211a470bc0540dc\": rpc error: code = NotFound desc = could not find container \"864c0a38e179893e8df109af747a3e011498a662d38e2ebc0211a470bc0540dc\": container with ID starting with 864c0a38e179893e8df109af747a3e011498a662d38e2ebc0211a470bc0540dc not found: ID does not exist" Jul 30 06:49:27.264618 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:27.264593 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77d84d85c6-wscbq"] Jul 30 06:49:27.268390 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:27.268370 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77d84d85c6-wscbq"] Jul 30 06:49:29.176169 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:29.176134 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1ba0bcb-7d3d-447e-8272-4910b15167d3" path="/var/lib/kubelet/pods/f1ba0bcb-7d3d-447e-8272-4910b15167d3/volumes" Jul 30 06:49:29.258918 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:29.258887 2575 generic.go:356] "Generic (PLEG): container finished" podID="bdbbfc08-6f67-4fda-b22a-4c00d352d582" containerID="a732760366ad111faf857eb0cefad8ae54370075b313df88dc8953280a2b1c51" exitCode=0 Jul 30 06:49:29.259076 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:29.258941 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-sflbp" event={"ID":"bdbbfc08-6f67-4fda-b22a-4c00d352d582","Type":"ContainerDied","Data":"a732760366ad111faf857eb0cefad8ae54370075b313df88dc8953280a2b1c51"} Jul 30 06:49:29.259278 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:29.259263 2575 scope.go:117] "RemoveContainer" containerID="a732760366ad111faf857eb0cefad8ae54370075b313df88dc8953280a2b1c51" Jul 30 06:49:30.264149 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:30.264116 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-sflbp" event={"ID":"bdbbfc08-6f67-4fda-b22a-4c00d352d582","Type":"ContainerStarted","Data":"efacdd8164d5746def03ef3139f62e3a37a196951b924f9e7f4bc55934390343"} Jul 30 06:49:31.528439 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:31.528401 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85cdd5d96b-gc469" podUID="e0f863f9-00cc-49fd-886a-518e3c977931" containerName="console" containerID="cri-o://8c8153ce57495c3d2da999b92dbbda83c9a6e5229b3a960b9fab4a885aa8d793" gracePeriod=15 Jul 30 06:49:31.814220 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:31.814192 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:49:31.923556 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:31.923524 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zh5xp\" (UniqueName: \"kubernetes.io/projected/e0f863f9-00cc-49fd-886a-518e3c977931-kube-api-access-zh5xp\") pod \"e0f863f9-00cc-49fd-886a-518e3c977931\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " Jul 30 06:49:31.923556 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:31.923567 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0f863f9-00cc-49fd-886a-518e3c977931-console-serving-cert\") pod \"e0f863f9-00cc-49fd-886a-518e3c977931\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " Jul 30 06:49:31.923824 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:31.923619 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-service-ca\") pod \"e0f863f9-00cc-49fd-886a-518e3c977931\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " Jul 30 06:49:31.923824 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:31.923691 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0f863f9-00cc-49fd-886a-518e3c977931-console-oauth-config\") pod \"e0f863f9-00cc-49fd-886a-518e3c977931\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " Jul 30 06:49:31.923824 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:31.923709 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-oauth-serving-cert\") pod \"e0f863f9-00cc-49fd-886a-518e3c977931\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " Jul 30 06:49:31.923824 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:31.923730 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-console-config\") pod \"e0f863f9-00cc-49fd-886a-518e3c977931\" (UID: \"e0f863f9-00cc-49fd-886a-518e3c977931\") " Jul 30 06:49:31.924185 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:31.924150 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-service-ca" (OuterVolumeSpecName: "service-ca") pod "e0f863f9-00cc-49fd-886a-518e3c977931" (UID: "e0f863f9-00cc-49fd-886a-518e3c977931"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:49:31.924185 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:31.924164 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-console-config" (OuterVolumeSpecName: "console-config") pod "e0f863f9-00cc-49fd-886a-518e3c977931" (UID: "e0f863f9-00cc-49fd-886a-518e3c977931"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:49:31.924344 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:31.924154 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e0f863f9-00cc-49fd-886a-518e3c977931" (UID: "e0f863f9-00cc-49fd-886a-518e3c977931"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:49:31.926161 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:31.926132 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0f863f9-00cc-49fd-886a-518e3c977931-kube-api-access-zh5xp" (OuterVolumeSpecName: "kube-api-access-zh5xp") pod "e0f863f9-00cc-49fd-886a-518e3c977931" (UID: "e0f863f9-00cc-49fd-886a-518e3c977931"). InnerVolumeSpecName "kube-api-access-zh5xp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 06:49:31.926247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:31.926170 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0f863f9-00cc-49fd-886a-518e3c977931-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e0f863f9-00cc-49fd-886a-518e3c977931" (UID: "e0f863f9-00cc-49fd-886a-518e3c977931"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:49:31.926247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:31.926205 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0f863f9-00cc-49fd-886a-518e3c977931-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e0f863f9-00cc-49fd-886a-518e3c977931" (UID: "e0f863f9-00cc-49fd-886a-518e3c977931"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:49:32.024952 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:32.024915 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-service-ca\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:49:32.024952 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:32.024946 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e0f863f9-00cc-49fd-886a-518e3c977931-console-oauth-config\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:49:32.024952 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:32.024958 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-oauth-serving-cert\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:49:32.025167 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:32.024966 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e0f863f9-00cc-49fd-886a-518e3c977931-console-config\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:49:32.025167 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:32.024975 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh5xp\" (UniqueName: \"kubernetes.io/projected/e0f863f9-00cc-49fd-886a-518e3c977931-kube-api-access-zh5xp\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:49:32.025167 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:32.024983 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e0f863f9-00cc-49fd-886a-518e3c977931-console-serving-cert\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:49:32.273341 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:32.273308 2575 generic.go:356] "Generic (PLEG): container finished" podID="e0f863f9-00cc-49fd-886a-518e3c977931" containerID="8c8153ce57495c3d2da999b92dbbda83c9a6e5229b3a960b9fab4a885aa8d793" exitCode=0 Jul 30 06:49:32.273507 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:32.273355 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cdd5d96b-gc469" Jul 30 06:49:32.273507 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:32.273399 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cdd5d96b-gc469" event={"ID":"e0f863f9-00cc-49fd-886a-518e3c977931","Type":"ContainerDied","Data":"8c8153ce57495c3d2da999b92dbbda83c9a6e5229b3a960b9fab4a885aa8d793"} Jul 30 06:49:32.273507 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:32.273452 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cdd5d96b-gc469" event={"ID":"e0f863f9-00cc-49fd-886a-518e3c977931","Type":"ContainerDied","Data":"22449c68d1c8c77b34c90eb428bf1d585d1e3e30b4ddf62fdd35432bb59b436b"} Jul 30 06:49:32.273507 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:32.273478 2575 scope.go:117] "RemoveContainer" containerID="8c8153ce57495c3d2da999b92dbbda83c9a6e5229b3a960b9fab4a885aa8d793" Jul 30 06:49:32.282518 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:32.282500 2575 scope.go:117] "RemoveContainer" containerID="8c8153ce57495c3d2da999b92dbbda83c9a6e5229b3a960b9fab4a885aa8d793" Jul 30 06:49:32.282798 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:49:32.282781 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c8153ce57495c3d2da999b92dbbda83c9a6e5229b3a960b9fab4a885aa8d793\": container with ID starting with 8c8153ce57495c3d2da999b92dbbda83c9a6e5229b3a960b9fab4a885aa8d793 not found: ID does not exist" containerID="8c8153ce57495c3d2da999b92dbbda83c9a6e5229b3a960b9fab4a885aa8d793" Jul 30 06:49:32.282849 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:32.282808 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c8153ce57495c3d2da999b92dbbda83c9a6e5229b3a960b9fab4a885aa8d793"} err="failed to get container status \"8c8153ce57495c3d2da999b92dbbda83c9a6e5229b3a960b9fab4a885aa8d793\": rpc error: code = NotFound desc = could not find container \"8c8153ce57495c3d2da999b92dbbda83c9a6e5229b3a960b9fab4a885aa8d793\": container with ID starting with 8c8153ce57495c3d2da999b92dbbda83c9a6e5229b3a960b9fab4a885aa8d793 not found: ID does not exist" Jul 30 06:49:32.293244 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:32.293213 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85cdd5d96b-gc469"] Jul 30 06:49:32.296842 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:32.296821 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85cdd5d96b-gc469"] Jul 30 06:49:33.175201 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:49:33.175174 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0f863f9-00cc-49fd-886a-518e3c977931" path="/var/lib/kubelet/pods/e0f863f9-00cc-49fd-886a-518e3c977931/volumes" Jul 30 06:50:13.157977 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:13.157936 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs\") pod \"network-metrics-daemon-7tcdf\" (UID: \"2a6fa524-6766-4ecd-a86a-5f26f29a72b2\") " pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:50:13.160398 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:13.160374 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a6fa524-6766-4ecd-a86a-5f26f29a72b2-metrics-certs\") pod \"network-metrics-daemon-7tcdf\" (UID: \"2a6fa524-6766-4ecd-a86a-5f26f29a72b2\") " pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:50:13.275216 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:13.275184 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8tfrn\"" Jul 30 06:50:13.283372 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:13.283353 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7tcdf" Jul 30 06:50:13.414762 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:13.414626 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7tcdf"] Jul 30 06:50:13.417644 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:50:13.417619 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a6fa524_6766_4ecd_a86a_5f26f29a72b2.slice/crio-1c6a287ad719a21f5ed3ed30a3ce810d7da5ae164c4e73cbba33fe77ec02bc0c WatchSource:0}: Error finding container 1c6a287ad719a21f5ed3ed30a3ce810d7da5ae164c4e73cbba33fe77ec02bc0c: Status 404 returned error can't find the container with id 1c6a287ad719a21f5ed3ed30a3ce810d7da5ae164c4e73cbba33fe77ec02bc0c Jul 30 06:50:13.430122 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:13.430096 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7tcdf" event={"ID":"2a6fa524-6766-4ecd-a86a-5f26f29a72b2","Type":"ContainerStarted","Data":"1c6a287ad719a21f5ed3ed30a3ce810d7da5ae164c4e73cbba33fe77ec02bc0c"} Jul 30 06:50:14.435522 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:14.435482 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7tcdf" event={"ID":"2a6fa524-6766-4ecd-a86a-5f26f29a72b2","Type":"ContainerStarted","Data":"65f4222ecd5e2faebf980566db854a936ba503d74a8862d4f95bbbee206e638d"} Jul 30 06:50:14.435887 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:14.435538 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7tcdf" event={"ID":"2a6fa524-6766-4ecd-a86a-5f26f29a72b2","Type":"ContainerStarted","Data":"f03be04f600d46aae7aca0a03a9e388c7e3159391fe2b6d07e79b81c64e16f27"} Jul 30 06:50:14.451621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:14.451574 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7tcdf" podStartSLOduration=374.650635657 podStartE2EDuration="6m15.451559834s" totalImagesPullingTime="800.924177ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:50:14.450248962 +0000 UTC m=+375.807253125" watchObservedRunningTime="2026-07-30 06:50:14.451559834 +0000 UTC m=+375.808563996" Jul 30 06:50:22.454361 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:22.454326 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 06:50:22.454840 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:22.454619 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="alertmanager" containerID="cri-o://09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376" gracePeriod=120 Jul 30 06:50:22.454840 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:22.454725 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="config-reloader" containerID="cri-o://cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b" gracePeriod=120 Jul 30 06:50:22.454840 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:22.454667 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="kube-rbac-proxy" containerID="cri-o://015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5" gracePeriod=120 Jul 30 06:50:22.454840 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:22.454696 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="prom-label-proxy" containerID="cri-o://b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20" gracePeriod=120 Jul 30 06:50:22.455051 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:22.454706 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="kube-rbac-proxy-web" containerID="cri-o://f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20" gracePeriod=120 Jul 30 06:50:22.455051 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:22.454706 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="kube-rbac-proxy-metric" containerID="cri-o://01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f" gracePeriod=120 Jul 30 06:50:23.473138 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.473107 2575 generic.go:356] "Generic (PLEG): container finished" podID="96d17342-cda6-4016-b1fb-8f3378febafc" containerID="b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20" exitCode=0 Jul 30 06:50:23.473138 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.473130 2575 generic.go:356] "Generic (PLEG): container finished" podID="96d17342-cda6-4016-b1fb-8f3378febafc" containerID="015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5" exitCode=0 Jul 30 06:50:23.473138 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.473136 2575 generic.go:356] "Generic (PLEG): container finished" podID="96d17342-cda6-4016-b1fb-8f3378febafc" containerID="cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b" exitCode=0 Jul 30 06:50:23.473138 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.473143 2575 generic.go:356] "Generic (PLEG): container finished" podID="96d17342-cda6-4016-b1fb-8f3378febafc" containerID="09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376" exitCode=0 Jul 30 06:50:23.473601 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.473176 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96d17342-cda6-4016-b1fb-8f3378febafc","Type":"ContainerDied","Data":"b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20"} Jul 30 06:50:23.473601 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.473237 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96d17342-cda6-4016-b1fb-8f3378febafc","Type":"ContainerDied","Data":"015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5"} Jul 30 06:50:23.473601 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.473267 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96d17342-cda6-4016-b1fb-8f3378febafc","Type":"ContainerDied","Data":"cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b"} Jul 30 06:50:23.473601 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.473291 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96d17342-cda6-4016-b1fb-8f3378febafc","Type":"ContainerDied","Data":"09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376"} Jul 30 06:50:23.695193 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.695172 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:23.845091 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.844993 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-config-volume\") pod \"96d17342-cda6-4016-b1fb-8f3378febafc\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " Jul 30 06:50:23.845091 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.845033 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-cluster-tls-config\") pod \"96d17342-cda6-4016-b1fb-8f3378febafc\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " Jul 30 06:50:23.845091 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.845080 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy\") pod \"96d17342-cda6-4016-b1fb-8f3378febafc\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " Jul 30 06:50:23.845371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.845110 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-web-config\") pod \"96d17342-cda6-4016-b1fb-8f3378febafc\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " Jul 30 06:50:23.845371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.845136 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96d17342-cda6-4016-b1fb-8f3378febafc-alertmanager-trusted-ca-bundle\") pod \"96d17342-cda6-4016-b1fb-8f3378febafc\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " Jul 30 06:50:23.845371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.845165 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/96d17342-cda6-4016-b1fb-8f3378febafc-tls-assets\") pod \"96d17342-cda6-4016-b1fb-8f3378febafc\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " Jul 30 06:50:23.845371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.845195 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/96d17342-cda6-4016-b1fb-8f3378febafc-config-out\") pod \"96d17342-cda6-4016-b1fb-8f3378febafc\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " Jul 30 06:50:23.845371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.845228 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/96d17342-cda6-4016-b1fb-8f3378febafc-alertmanager-main-db\") pod \"96d17342-cda6-4016-b1fb-8f3378febafc\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " Jul 30 06:50:23.845371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.845263 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"96d17342-cda6-4016-b1fb-8f3378febafc\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " Jul 30 06:50:23.845371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.845290 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96d17342-cda6-4016-b1fb-8f3378febafc-metrics-client-ca\") pod \"96d17342-cda6-4016-b1fb-8f3378febafc\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " Jul 30 06:50:23.845371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.845323 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvfl8\" (UniqueName: \"kubernetes.io/projected/96d17342-cda6-4016-b1fb-8f3378febafc-kube-api-access-lvfl8\") pod \"96d17342-cda6-4016-b1fb-8f3378febafc\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " Jul 30 06:50:23.845371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.845350 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy-web\") pod \"96d17342-cda6-4016-b1fb-8f3378febafc\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " Jul 30 06:50:23.845371 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.845376 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-main-tls\") pod \"96d17342-cda6-4016-b1fb-8f3378febafc\" (UID: \"96d17342-cda6-4016-b1fb-8f3378febafc\") " Jul 30 06:50:23.845918 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.845514 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96d17342-cda6-4016-b1fb-8f3378febafc-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "96d17342-cda6-4016-b1fb-8f3378febafc" (UID: "96d17342-cda6-4016-b1fb-8f3378febafc"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:50:23.845918 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.845622 2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/96d17342-cda6-4016-b1fb-8f3378febafc-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:50:23.846421 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.846368 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96d17342-cda6-4016-b1fb-8f3378febafc-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "96d17342-cda6-4016-b1fb-8f3378febafc" (UID: "96d17342-cda6-4016-b1fb-8f3378febafc"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 06:50:23.846562 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.846531 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96d17342-cda6-4016-b1fb-8f3378febafc-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "96d17342-cda6-4016-b1fb-8f3378febafc" (UID: "96d17342-cda6-4016-b1fb-8f3378febafc"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:50:23.848567 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.848343 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-config-volume" (OuterVolumeSpecName: "config-volume") pod "96d17342-cda6-4016-b1fb-8f3378febafc" (UID: "96d17342-cda6-4016-b1fb-8f3378febafc"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:50:23.848980 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.848951 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "96d17342-cda6-4016-b1fb-8f3378febafc" (UID: "96d17342-cda6-4016-b1fb-8f3378febafc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:50:23.849088 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.849007 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96d17342-cda6-4016-b1fb-8f3378febafc-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "96d17342-cda6-4016-b1fb-8f3378febafc" (UID: "96d17342-cda6-4016-b1fb-8f3378febafc"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 06:50:23.849145 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.849085 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "96d17342-cda6-4016-b1fb-8f3378febafc" (UID: "96d17342-cda6-4016-b1fb-8f3378febafc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:50:23.849188 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.849147 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "96d17342-cda6-4016-b1fb-8f3378febafc" (UID: "96d17342-cda6-4016-b1fb-8f3378febafc"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:50:23.849517 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.849491 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96d17342-cda6-4016-b1fb-8f3378febafc-config-out" (OuterVolumeSpecName: "config-out") pod "96d17342-cda6-4016-b1fb-8f3378febafc" (UID: "96d17342-cda6-4016-b1fb-8f3378febafc"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 06:50:23.849943 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.849923 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96d17342-cda6-4016-b1fb-8f3378febafc-kube-api-access-lvfl8" (OuterVolumeSpecName: "kube-api-access-lvfl8") pod "96d17342-cda6-4016-b1fb-8f3378febafc" (UID: "96d17342-cda6-4016-b1fb-8f3378febafc"). InnerVolumeSpecName "kube-api-access-lvfl8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 06:50:23.850032 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.850009 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "96d17342-cda6-4016-b1fb-8f3378febafc" (UID: "96d17342-cda6-4016-b1fb-8f3378febafc"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:50:23.853161 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.853142 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "96d17342-cda6-4016-b1fb-8f3378febafc" (UID: "96d17342-cda6-4016-b1fb-8f3378febafc"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:50:23.859438 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.859419 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-web-config" (OuterVolumeSpecName: "web-config") pod "96d17342-cda6-4016-b1fb-8f3378febafc" (UID: "96d17342-cda6-4016-b1fb-8f3378febafc"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:50:23.946730 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.946693 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:50:23.946901 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.946750 2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-web-config\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:50:23.946901 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.946762 2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/96d17342-cda6-4016-b1fb-8f3378febafc-tls-assets\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:50:23.946901 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.946770 2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/96d17342-cda6-4016-b1fb-8f3378febafc-config-out\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:50:23.946901 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.946779 2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/96d17342-cda6-4016-b1fb-8f3378febafc-alertmanager-main-db\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:50:23.946901 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.946788 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:50:23.946901 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.946801 2575 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96d17342-cda6-4016-b1fb-8f3378febafc-metrics-client-ca\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:50:23.946901 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.946814 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvfl8\" (UniqueName: \"kubernetes.io/projected/96d17342-cda6-4016-b1fb-8f3378febafc-kube-api-access-lvfl8\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:50:23.946901 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.946823 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:50:23.946901 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.946834 2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-secret-alertmanager-main-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:50:23.946901 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.946842 2575 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-config-volume\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:50:23.946901 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:23.946850 2575 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/96d17342-cda6-4016-b1fb-8f3378febafc-cluster-tls-config\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:50:24.479982 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.479946 2575 generic.go:356] "Generic (PLEG): container finished" podID="96d17342-cda6-4016-b1fb-8f3378febafc" containerID="01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f" exitCode=0 Jul 30 06:50:24.479982 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.479971 2575 generic.go:356] "Generic (PLEG): container finished" podID="96d17342-cda6-4016-b1fb-8f3378febafc" containerID="f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20" exitCode=0 Jul 30 06:50:24.480460 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.480032 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.480460 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.480034 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96d17342-cda6-4016-b1fb-8f3378febafc","Type":"ContainerDied","Data":"01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f"} Jul 30 06:50:24.480460 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.480118 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96d17342-cda6-4016-b1fb-8f3378febafc","Type":"ContainerDied","Data":"f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20"} Jul 30 06:50:24.480460 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.480161 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"96d17342-cda6-4016-b1fb-8f3378febafc","Type":"ContainerDied","Data":"d44b841b2b076500f27733796ffc2f0cc238364a584157b1d74dc8014d8c3a7d"} Jul 30 06:50:24.480460 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.480191 2575 scope.go:117] "RemoveContainer" containerID="b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20" Jul 30 06:50:24.488373 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.488357 2575 scope.go:117] "RemoveContainer" containerID="01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f" Jul 30 06:50:24.495607 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.495589 2575 scope.go:117] "RemoveContainer" containerID="015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5" Jul 30 06:50:24.502711 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.502695 2575 scope.go:117] "RemoveContainer" containerID="f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20" Jul 30 06:50:24.503882 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.503863 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 06:50:24.507574 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.507556 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 06:50:24.512129 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.512111 2575 scope.go:117] "RemoveContainer" containerID="cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b" Jul 30 06:50:24.519211 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.519190 2575 scope.go:117] "RemoveContainer" containerID="09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376" Jul 30 06:50:24.526237 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.526218 2575 scope.go:117] "RemoveContainer" containerID="b84ac8fa11a7c21ab84eb0e18d45c429c044aae36538cc22e02ce31edd45d784" Jul 30 06:50:24.533146 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.533122 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 06:50:24.533417 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.533404 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="config-reloader" Jul 30 06:50:24.533485 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.533420 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="config-reloader" Jul 30 06:50:24.533485 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.533429 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="alertmanager" Jul 30 06:50:24.533485 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.533435 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="alertmanager" Jul 30 06:50:24.533485 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.533451 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="init-config-reloader" Jul 30 06:50:24.533485 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.533459 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="init-config-reloader" Jul 30 06:50:24.533485 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.533469 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="kube-rbac-proxy" Jul 30 06:50:24.533485 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.533477 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="kube-rbac-proxy" Jul 30 06:50:24.533485 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.533486 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="prom-label-proxy" Jul 30 06:50:24.534017 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.533493 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="prom-label-proxy" Jul 30 06:50:24.534017 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.533503 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="kube-rbac-proxy-metric" Jul 30 06:50:24.534017 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.533508 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="kube-rbac-proxy-metric" Jul 30 06:50:24.534017 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.533516 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1ba0bcb-7d3d-447e-8272-4910b15167d3" containerName="registry" Jul 30 06:50:24.534017 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.533521 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1ba0bcb-7d3d-447e-8272-4910b15167d3" containerName="registry" Jul 30 06:50:24.534017 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.533531 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0f863f9-00cc-49fd-886a-518e3c977931" containerName="console" Jul 30 06:50:24.534017 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.534005 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0f863f9-00cc-49fd-886a-518e3c977931" containerName="console" Jul 30 06:50:24.534340 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.534021 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="kube-rbac-proxy-web" Jul 30 06:50:24.534340 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.534032 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="kube-rbac-proxy-web" Jul 30 06:50:24.537727 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.534750 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="kube-rbac-proxy-metric" Jul 30 06:50:24.537727 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.534779 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="config-reloader" Jul 30 06:50:24.537727 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.534791 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="kube-rbac-proxy" Jul 30 06:50:24.537727 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.534809 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="alertmanager" Jul 30 06:50:24.537727 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.534818 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1ba0bcb-7d3d-447e-8272-4910b15167d3" containerName="registry" Jul 30 06:50:24.537727 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.534834 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="kube-rbac-proxy-web" Jul 30 06:50:24.537727 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.534844 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0f863f9-00cc-49fd-886a-518e3c977931" containerName="console" Jul 30 06:50:24.537727 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.534851 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" containerName="prom-label-proxy" Jul 30 06:50:24.538547 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.538528 2575 scope.go:117] "RemoveContainer" containerID="b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20" Jul 30 06:50:24.539920 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:50:24.539887 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20\": container with ID starting with b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20 not found: ID does not exist" containerID="b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20" Jul 30 06:50:24.540016 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.539933 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20"} err="failed to get container status \"b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20\": rpc error: code = NotFound desc = could not find container \"b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20\": container with ID starting with b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20 not found: ID does not exist" Jul 30 06:50:24.540016 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.539954 2575 scope.go:117] "RemoveContainer" containerID="01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f" Jul 30 06:50:24.540569 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:50:24.540548 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f\": container with ID starting with 01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f not found: ID does not exist" containerID="01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f" Jul 30 06:50:24.540649 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.540574 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f"} err="failed to get container status \"01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f\": rpc error: code = NotFound desc = could not find container \"01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f\": container with ID starting with 01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f not found: ID does not exist" Jul 30 06:50:24.540649 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.540596 2575 scope.go:117] "RemoveContainer" containerID="015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5" Jul 30 06:50:24.540649 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.540603 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.541175 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:50:24.541153 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5\": container with ID starting with 015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5 not found: ID does not exist" containerID="015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5" Jul 30 06:50:24.541323 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.541184 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5"} err="failed to get container status \"015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5\": rpc error: code = NotFound desc = could not find container \"015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5\": container with ID starting with 015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5 not found: ID does not exist" Jul 30 06:50:24.541323 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.541202 2575 scope.go:117] "RemoveContainer" containerID="f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20" Jul 30 06:50:24.541449 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:50:24.541432 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20\": container with ID starting with f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20 not found: ID does not exist" containerID="f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20" Jul 30 06:50:24.541502 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.541453 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20"} err="failed to get container status \"f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20\": rpc error: code = NotFound desc = could not find container \"f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20\": container with ID starting with f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20 not found: ID does not exist" Jul 30 06:50:24.541502 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.541466 2575 scope.go:117] "RemoveContainer" containerID="cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b" Jul 30 06:50:24.541743 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:50:24.541719 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b\": container with ID starting with cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b not found: ID does not exist" containerID="cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b" Jul 30 06:50:24.541822 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.541771 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b"} err="failed to get container status \"cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b\": rpc error: code = NotFound desc = could not find container \"cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b\": container with ID starting with cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b not found: ID does not exist" Jul 30 06:50:24.541822 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.541796 2575 scope.go:117] "RemoveContainer" containerID="09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376" Jul 30 06:50:24.542029 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:50:24.542014 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376\": container with ID starting with 09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376 not found: ID does not exist" containerID="09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376" Jul 30 06:50:24.542071 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.542033 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376"} err="failed to get container status \"09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376\": rpc error: code = NotFound desc = could not find container \"09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376\": container with ID starting with 09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376 not found: ID does not exist" Jul 30 06:50:24.542071 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.542046 2575 scope.go:117] "RemoveContainer" containerID="b84ac8fa11a7c21ab84eb0e18d45c429c044aae36538cc22e02ce31edd45d784" Jul 30 06:50:24.542339 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:50:24.542323 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b84ac8fa11a7c21ab84eb0e18d45c429c044aae36538cc22e02ce31edd45d784\": container with ID starting with b84ac8fa11a7c21ab84eb0e18d45c429c044aae36538cc22e02ce31edd45d784 not found: ID does not exist" containerID="b84ac8fa11a7c21ab84eb0e18d45c429c044aae36538cc22e02ce31edd45d784" Jul 30 06:50:24.542409 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.542346 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b84ac8fa11a7c21ab84eb0e18d45c429c044aae36538cc22e02ce31edd45d784"} err="failed to get container status \"b84ac8fa11a7c21ab84eb0e18d45c429c044aae36538cc22e02ce31edd45d784\": rpc error: code = NotFound desc = could not find container \"b84ac8fa11a7c21ab84eb0e18d45c429c044aae36538cc22e02ce31edd45d784\": container with ID starting with b84ac8fa11a7c21ab84eb0e18d45c429c044aae36538cc22e02ce31edd45d784 not found: ID does not exist" Jul 30 06:50:24.542409 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.542366 2575 scope.go:117] "RemoveContainer" containerID="b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20" Jul 30 06:50:24.542669 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.542623 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20"} err="failed to get container status \"b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20\": rpc error: code = NotFound desc = could not find container \"b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20\": container with ID starting with b0dd0f5f84bb4321c67580907dd91ff96be0168ca3456ec6ed79b467c8fc5b20 not found: ID does not exist" Jul 30 06:50:24.542719 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.542673 2575 scope.go:117] "RemoveContainer" containerID="01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f" Jul 30 06:50:24.542960 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.542941 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f"} err="failed to get container status \"01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f\": rpc error: code = NotFound desc = could not find container \"01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f\": container with ID starting with 01f7e16209dd2dd899348f8b2adcbf08d904231a3253cddbeda6ab6cbda5a10f not found: ID does not exist" Jul 30 06:50:24.543053 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.542969 2575 scope.go:117] "RemoveContainer" containerID="015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5" Jul 30 06:50:24.543053 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.542942 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 30 06:50:24.543164 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.543123 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 30 06:50:24.543210 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.543190 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 30 06:50:24.543308 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.543279 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5"} err="failed to get container status \"015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5\": rpc error: code = NotFound desc = could not find container \"015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5\": container with ID starting with 015ad3e148e80de92e26dbb6c8c74dbb5a39530278fbf14b2265e03b163621e5 not found: ID does not exist" Jul 30 06:50:24.543308 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.543308 2575 scope.go:117] "RemoveContainer" containerID="f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20" Jul 30 06:50:24.543507 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.543391 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 30 06:50:24.543579 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.543533 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 30 06:50:24.543636 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.543590 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20"} err="failed to get container status \"f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20\": rpc error: code = NotFound desc = could not find container \"f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20\": container with ID starting with f55e7d55d7b6a4a6abf8e4692bbc4ebb9922ffdc28306e1bfc1856f146428c20 not found: ID does not exist" Jul 30 06:50:24.543636 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.543606 2575 scope.go:117] "RemoveContainer" containerID="cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b" Jul 30 06:50:24.543783 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.543726 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 30 06:50:24.543836 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.543730 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 30 06:50:24.543836 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.543797 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 30 06:50:24.543933 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.543875 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jhc44\"" Jul 30 06:50:24.543972 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.543929 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b"} err="failed to get container status \"cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b\": rpc error: code = NotFound desc = could not find container \"cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b\": container with ID starting with cdaae7ad6023aec38c84055b58c069bf68a43c279453dc277f553fcf7e43eb9b not found: ID does not exist" Jul 30 06:50:24.543972 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.543947 2575 scope.go:117] "RemoveContainer" containerID="09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376" Jul 30 06:50:24.544282 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.544255 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376"} err="failed to get container status \"09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376\": rpc error: code = NotFound desc = could not find container \"09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376\": container with ID starting with 09aefc6015954d0faed02da76519db59723dbb430a473cc4153f7caa95ae8376 not found: ID does not exist" Jul 30 06:50:24.544370 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.544284 2575 scope.go:117] "RemoveContainer" containerID="b84ac8fa11a7c21ab84eb0e18d45c429c044aae36538cc22e02ce31edd45d784" Jul 30 06:50:24.544554 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.544531 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b84ac8fa11a7c21ab84eb0e18d45c429c044aae36538cc22e02ce31edd45d784"} err="failed to get container status \"b84ac8fa11a7c21ab84eb0e18d45c429c044aae36538cc22e02ce31edd45d784\": rpc error: code = NotFound desc = could not find container \"b84ac8fa11a7c21ab84eb0e18d45c429c044aae36538cc22e02ce31edd45d784\": container with ID starting with b84ac8fa11a7c21ab84eb0e18d45c429c044aae36538cc22e02ce31edd45d784 not found: ID does not exist" Jul 30 06:50:24.548030 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.547943 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 30 06:50:24.550293 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.550271 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 06:50:24.654077 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.654047 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.654247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.654082 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.654247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.654105 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.654247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.654129 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.654247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.654189 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-web-config\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.654247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.654232 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv2wh\" (UniqueName: \"kubernetes.io/projected/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-kube-api-access-bv2wh\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.654428 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.654262 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.654428 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.654280 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.654428 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.654301 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-config-out\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.654428 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.654319 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.654428 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.654334 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-config-volume\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.654428 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.654377 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.654428 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.654423 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.755013 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.754933 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.755013 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.754976 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.755205 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.755015 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.755205 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.755036 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.755205 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.755053 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.755205 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.755070 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.755205 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.755160 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-web-config\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.755205 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.755200 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv2wh\" (UniqueName: \"kubernetes.io/projected/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-kube-api-access-bv2wh\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.755527 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.755233 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.755527 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.755256 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.755527 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.755288 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-config-out\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.755527 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.755319 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.755527 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.755345 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-config-volume\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.755805 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.755568 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.755963 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.755936 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.756886 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.756856 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.758563 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.758539 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.758770 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.758748 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-web-config\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.758853 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.758825 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.759012 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.758993 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.759092 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.759074 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-config-out\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.759396 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.759369 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.759506 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.759423 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.759593 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.759577 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-config-volume\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.760006 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.759990 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.764142 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.764114 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv2wh\" (UniqueName: \"kubernetes.io/projected/ed5ae34f-c018-4cb2-90ad-bb172a6be9bd-kube-api-access-bv2wh\") pod \"alertmanager-main-0\" (UID: \"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.851135 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.851094 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 30 06:50:24.990094 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:24.989982 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 30 06:50:24.992452 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:50:24.992426 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded5ae34f_c018_4cb2_90ad_bb172a6be9bd.slice/crio-0540ebd4a75a8dd02800b958e5177f4c20c7eba7e4c4457c0776e462427d3454 WatchSource:0}: Error finding container 0540ebd4a75a8dd02800b958e5177f4c20c7eba7e4c4457c0776e462427d3454: Status 404 returned error can't find the container with id 0540ebd4a75a8dd02800b958e5177f4c20c7eba7e4c4457c0776e462427d3454 Jul 30 06:50:25.175347 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:25.175311 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96d17342-cda6-4016-b1fb-8f3378febafc" path="/var/lib/kubelet/pods/96d17342-cda6-4016-b1fb-8f3378febafc/volumes" Jul 30 06:50:25.485002 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:25.484965 2575 generic.go:356] "Generic (PLEG): container finished" podID="ed5ae34f-c018-4cb2-90ad-bb172a6be9bd" containerID="314377fb19c082c0e497a3c8ddf9010b69e64d610a76ed5b8cba31beec184104" exitCode=0 Jul 30 06:50:25.485444 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:25.485049 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd","Type":"ContainerDied","Data":"314377fb19c082c0e497a3c8ddf9010b69e64d610a76ed5b8cba31beec184104"} Jul 30 06:50:25.485444 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:25.485115 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd","Type":"ContainerStarted","Data":"0540ebd4a75a8dd02800b958e5177f4c20c7eba7e4c4457c0776e462427d3454"} Jul 30 06:50:26.483263 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.483218 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-76cb47f688-kspv5"] Jul 30 06:50:26.486124 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.486097 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.488554 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.488527 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 30 06:50:26.488845 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.488814 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 30 06:50:26.488962 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.488858 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 30 06:50:26.489047 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.489023 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-vcqgb\"" Jul 30 06:50:26.489166 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.489145 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 30 06:50:26.489342 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.489324 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 30 06:50:26.498853 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.497033 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 30 06:50:26.501627 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.501596 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76cb47f688-kspv5"] Jul 30 06:50:26.504249 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.503346 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd","Type":"ContainerStarted","Data":"9c69da4df843b1a0b79b2a50edccbdcb1f603cfc040d2576d476a082f46dd434"} Jul 30 06:50:26.504249 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.503447 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd","Type":"ContainerStarted","Data":"9f72eadc195201244aa37ab16022f6c7ca6bf2aadb8b8c0d27ee753c89d81aff"} Jul 30 06:50:26.504249 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.503511 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd","Type":"ContainerStarted","Data":"673e489c10d66f5110b06b7dbb3c2a19973ae3adcabbec64a34c7f889eccb87b"} Jul 30 06:50:26.504249 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.503562 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd","Type":"ContainerStarted","Data":"bab84ed605a91c6736933a05af78a1d0e8678480315c4cb2d8c030af34557dce"} Jul 30 06:50:26.504249 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.503617 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd","Type":"ContainerStarted","Data":"f84eb4630713898eff36bb8a9071abbd38a38f2d4c1f7dc0f0002057d0e09790"} Jul 30 06:50:26.504249 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.503696 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ed5ae34f-c018-4cb2-90ad-bb172a6be9bd","Type":"ContainerStarted","Data":"2a8b690f6d86340a83c7e4a840e480258532b8ecfc1d0bfb57cbdbc9cb80ae8b"} Jul 30 06:50:26.543956 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.543900 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.529051374 podStartE2EDuration="2.543883189s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.831815ms" isStatefulPod=false podCreationTimestamp="2026-07-30 06:50:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:50:26.542467446 +0000 UTC m=+387.899471608" watchObservedRunningTime="2026-07-30 06:50:26.543883189 +0000 UTC m=+387.900887351" Jul 30 06:50:26.568350 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.568305 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d44e3ab-79a3-4646-88d6-446242d7500d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.568513 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.568398 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d44e3ab-79a3-4646-88d6-446242d7500d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.568513 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.568435 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0d44e3ab-79a3-4646-88d6-446242d7500d-telemeter-client-tls\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.568513 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.568478 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d44e3ab-79a3-4646-88d6-446242d7500d-metrics-client-ca\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.568809 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.568526 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0d44e3ab-79a3-4646-88d6-446242d7500d-secret-telemeter-client\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.568809 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.568583 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d44e3ab-79a3-4646-88d6-446242d7500d-serving-certs-ca-bundle\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.568809 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.568624 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0d44e3ab-79a3-4646-88d6-446242d7500d-federate-client-tls\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.568809 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.568649 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t87vf\" (UniqueName: \"kubernetes.io/projected/0d44e3ab-79a3-4646-88d6-446242d7500d-kube-api-access-t87vf\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.669919 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.669875 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d44e3ab-79a3-4646-88d6-446242d7500d-serving-certs-ca-bundle\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.669919 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.669914 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0d44e3ab-79a3-4646-88d6-446242d7500d-federate-client-tls\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.670165 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.670021 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t87vf\" (UniqueName: \"kubernetes.io/projected/0d44e3ab-79a3-4646-88d6-446242d7500d-kube-api-access-t87vf\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.670165 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.670086 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d44e3ab-79a3-4646-88d6-446242d7500d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.670697 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.670646 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d44e3ab-79a3-4646-88d6-446242d7500d-serving-certs-ca-bundle\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.670847 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.670672 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d44e3ab-79a3-4646-88d6-446242d7500d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.670847 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.670796 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0d44e3ab-79a3-4646-88d6-446242d7500d-telemeter-client-tls\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.670847 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.670823 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d44e3ab-79a3-4646-88d6-446242d7500d-metrics-client-ca\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.671011 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.670888 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0d44e3ab-79a3-4646-88d6-446242d7500d-secret-telemeter-client\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.671067 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.671017 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0d44e3ab-79a3-4646-88d6-446242d7500d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.671964 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.671877 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d44e3ab-79a3-4646-88d6-446242d7500d-metrics-client-ca\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.673760 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.673736 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0d44e3ab-79a3-4646-88d6-446242d7500d-federate-client-tls\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.674176 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.674149 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0d44e3ab-79a3-4646-88d6-446242d7500d-telemeter-client-tls\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.674321 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.674299 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d44e3ab-79a3-4646-88d6-446242d7500d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.674819 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.674798 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0d44e3ab-79a3-4646-88d6-446242d7500d-secret-telemeter-client\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.677986 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.677965 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t87vf\" (UniqueName: \"kubernetes.io/projected/0d44e3ab-79a3-4646-88d6-446242d7500d-kube-api-access-t87vf\") pod \"telemeter-client-76cb47f688-kspv5\" (UID: \"0d44e3ab-79a3-4646-88d6-446242d7500d\") " pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.803601 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.803511 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" Jul 30 06:50:26.953974 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:26.953948 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76cb47f688-kspv5"] Jul 30 06:50:26.955556 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:50:26.955530 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d44e3ab_79a3_4646_88d6_446242d7500d.slice/crio-6c2746edaff92486b32f229366d0c34554c6c69b1054fc3bc0994b900b732587 WatchSource:0}: Error finding container 6c2746edaff92486b32f229366d0c34554c6c69b1054fc3bc0994b900b732587: Status 404 returned error can't find the container with id 6c2746edaff92486b32f229366d0c34554c6c69b1054fc3bc0994b900b732587 Jul 30 06:50:27.508931 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:27.508898 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" event={"ID":"0d44e3ab-79a3-4646-88d6-446242d7500d","Type":"ContainerStarted","Data":"6c2746edaff92486b32f229366d0c34554c6c69b1054fc3bc0994b900b732587"} Jul 30 06:50:29.519194 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:29.519150 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" event={"ID":"0d44e3ab-79a3-4646-88d6-446242d7500d","Type":"ContainerStarted","Data":"4a3d93e349448fe2d11d906ca893d2f823c5195ae501a8e147c47ea06a904ef0"} Jul 30 06:50:29.519607 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:29.519205 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" event={"ID":"0d44e3ab-79a3-4646-88d6-446242d7500d","Type":"ContainerStarted","Data":"3b21322096eee60f6b6054a10082542e59a05e2eafe3193644ed721536c8a8ee"} Jul 30 06:50:29.519607 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:29.519224 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" event={"ID":"0d44e3ab-79a3-4646-88d6-446242d7500d","Type":"ContainerStarted","Data":"b81405a7969e1a6720f02e6394c11d52399047f3d6de036e30a813506369a048"} Jul 30 06:50:29.542792 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:29.542744 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-76cb47f688-kspv5" podStartSLOduration=1.703205474 podStartE2EDuration="3.542728613s" totalImagesPullingTime="1.839523139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:50:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:50:29.539572953 +0000 UTC m=+390.896577116" watchObservedRunningTime="2026-07-30 06:50:29.542728613 +0000 UTC m=+390.899732775" Jul 30 06:50:30.367276 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.367240 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d447b797b-h2vgf"] Jul 30 06:50:30.369788 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.369765 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.380764 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.380737 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d447b797b-h2vgf"] Jul 30 06:50:30.501400 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.501355 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-trusted-ca-bundle\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.501756 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.501723 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-serving-cert\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.501899 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.501816 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c8sk\" (UniqueName: \"kubernetes.io/projected/4698337d-9c90-4e8b-90a0-c2ad585d1748-kube-api-access-6c8sk\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.501899 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.501875 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-oauth-config\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.502010 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.501909 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-oauth-serving-cert\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.502010 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.501949 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-config\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.502010 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.501998 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-service-ca\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.602827 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.602790 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-config\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.603339 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.602899 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-service-ca\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.603339 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.602974 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-trusted-ca-bundle\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.603339 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.603059 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-serving-cert\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.603339 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.603084 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c8sk\" (UniqueName: \"kubernetes.io/projected/4698337d-9c90-4e8b-90a0-c2ad585d1748-kube-api-access-6c8sk\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.603339 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.603174 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-oauth-config\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.603339 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.603232 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-oauth-serving-cert\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.603940 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.603915 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-service-ca\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.604097 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.604020 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-config\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.606893 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.604680 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-oauth-serving-cert\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.606893 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.604682 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-trusted-ca-bundle\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.606893 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.606388 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-oauth-config\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.606893 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.606839 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-serving-cert\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.610983 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.610956 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c8sk\" (UniqueName: \"kubernetes.io/projected/4698337d-9c90-4e8b-90a0-c2ad585d1748-kube-api-access-6c8sk\") pod \"console-7d447b797b-h2vgf\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.679927 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.679892 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:30.830156 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:30.830100 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d447b797b-h2vgf"] Jul 30 06:50:30.832781 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:50:30.832747 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4698337d_9c90_4e8b_90a0_c2ad585d1748.slice/crio-d6032c5638a8f353ca9679ce5a3ec6fb4c6b10400d930858d76e4e6137e13f85 WatchSource:0}: Error finding container d6032c5638a8f353ca9679ce5a3ec6fb4c6b10400d930858d76e4e6137e13f85: Status 404 returned error can't find the container with id d6032c5638a8f353ca9679ce5a3ec6fb4c6b10400d930858d76e4e6137e13f85 Jul 30 06:50:31.530174 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:31.530136 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d447b797b-h2vgf" event={"ID":"4698337d-9c90-4e8b-90a0-c2ad585d1748","Type":"ContainerStarted","Data":"fbb0bb9d843cf113e48114b74fe9c533f357af2228f7b3250b319b8ff3eda52d"} Jul 30 06:50:31.530467 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:31.530193 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d447b797b-h2vgf" event={"ID":"4698337d-9c90-4e8b-90a0-c2ad585d1748","Type":"ContainerStarted","Data":"d6032c5638a8f353ca9679ce5a3ec6fb4c6b10400d930858d76e4e6137e13f85"} Jul 30 06:50:31.548835 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:31.548796 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d447b797b-h2vgf" podStartSLOduration=1.548780729 podStartE2EDuration="1.548780729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:50:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:50:31.547195359 +0000 UTC m=+392.904199555" watchObservedRunningTime="2026-07-30 06:50:31.548780729 +0000 UTC m=+392.905784891" Jul 30 06:50:40.680600 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:40.680551 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:40.681048 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:40.680622 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:40.685512 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:40.685482 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:41.573329 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:41.573303 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:50:41.619769 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:50:41.619735 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cc4596cdf-vt55p"] Jul 30 06:51:06.640000 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:06.639937 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cc4596cdf-vt55p" podUID="b3e757c4-bb98-457a-9992-5b738f8a8249" containerName="console" containerID="cri-o://5e9defcdd924427f571a0518b7cc3b2244c0a246fb6914931e5ea3bb561eac8b" gracePeriod=15 Jul 30 06:51:06.874444 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:06.874420 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:51:06.996830 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:06.996742 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3e757c4-bb98-457a-9992-5b738f8a8249-console-serving-cert\") pod \"b3e757c4-bb98-457a-9992-5b738f8a8249\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " Jul 30 06:51:06.996830 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:06.996781 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkkn6\" (UniqueName: \"kubernetes.io/projected/b3e757c4-bb98-457a-9992-5b738f8a8249-kube-api-access-pkkn6\") pod \"b3e757c4-bb98-457a-9992-5b738f8a8249\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " Jul 30 06:51:06.996830 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:06.996800 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-oauth-serving-cert\") pod \"b3e757c4-bb98-457a-9992-5b738f8a8249\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " Jul 30 06:51:06.997078 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:06.996831 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3e757c4-bb98-457a-9992-5b738f8a8249-console-oauth-config\") pod \"b3e757c4-bb98-457a-9992-5b738f8a8249\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " Jul 30 06:51:06.997078 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:06.996865 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-service-ca\") pod \"b3e757c4-bb98-457a-9992-5b738f8a8249\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " Jul 30 06:51:06.997078 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:06.996926 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-trusted-ca-bundle\") pod \"b3e757c4-bb98-457a-9992-5b738f8a8249\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " Jul 30 06:51:06.997078 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:06.996965 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-console-config\") pod \"b3e757c4-bb98-457a-9992-5b738f8a8249\" (UID: \"b3e757c4-bb98-457a-9992-5b738f8a8249\") " Jul 30 06:51:06.997303 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:06.997226 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b3e757c4-bb98-457a-9992-5b738f8a8249" (UID: "b3e757c4-bb98-457a-9992-5b738f8a8249"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:51:06.997394 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:06.997285 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-service-ca" (OuterVolumeSpecName: "service-ca") pod "b3e757c4-bb98-457a-9992-5b738f8a8249" (UID: "b3e757c4-bb98-457a-9992-5b738f8a8249"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:51:06.997454 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:06.997377 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b3e757c4-bb98-457a-9992-5b738f8a8249" (UID: "b3e757c4-bb98-457a-9992-5b738f8a8249"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:51:06.997454 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:06.997437 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-console-config" (OuterVolumeSpecName: "console-config") pod "b3e757c4-bb98-457a-9992-5b738f8a8249" (UID: "b3e757c4-bb98-457a-9992-5b738f8a8249"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:51:06.999135 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:06.999106 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3e757c4-bb98-457a-9992-5b738f8a8249-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b3e757c4-bb98-457a-9992-5b738f8a8249" (UID: "b3e757c4-bb98-457a-9992-5b738f8a8249"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:51:06.999245 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:06.999168 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3e757c4-bb98-457a-9992-5b738f8a8249-kube-api-access-pkkn6" (OuterVolumeSpecName: "kube-api-access-pkkn6") pod "b3e757c4-bb98-457a-9992-5b738f8a8249" (UID: "b3e757c4-bb98-457a-9992-5b738f8a8249"). InnerVolumeSpecName "kube-api-access-pkkn6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 06:51:06.999245 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:06.999203 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3e757c4-bb98-457a-9992-5b738f8a8249-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b3e757c4-bb98-457a-9992-5b738f8a8249" (UID: "b3e757c4-bb98-457a-9992-5b738f8a8249"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:51:07.098500 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:07.098458 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b3e757c4-bb98-457a-9992-5b738f8a8249-console-oauth-config\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:51:07.098500 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:07.098490 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-service-ca\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:51:07.098500 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:07.098503 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-trusted-ca-bundle\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:51:07.098812 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:07.098513 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-console-config\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:51:07.098812 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:07.098525 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b3e757c4-bb98-457a-9992-5b738f8a8249-console-serving-cert\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:51:07.098812 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:07.098536 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkkn6\" (UniqueName: \"kubernetes.io/projected/b3e757c4-bb98-457a-9992-5b738f8a8249-kube-api-access-pkkn6\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:51:07.098812 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:07.098547 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b3e757c4-bb98-457a-9992-5b738f8a8249-oauth-serving-cert\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:51:07.668223 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:07.668187 2575 generic.go:356] "Generic (PLEG): container finished" podID="b3e757c4-bb98-457a-9992-5b738f8a8249" containerID="5e9defcdd924427f571a0518b7cc3b2244c0a246fb6914931e5ea3bb561eac8b" exitCode=0 Jul 30 06:51:07.668606 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:07.668228 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc4596cdf-vt55p" Jul 30 06:51:07.668606 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:07.668267 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc4596cdf-vt55p" event={"ID":"b3e757c4-bb98-457a-9992-5b738f8a8249","Type":"ContainerDied","Data":"5e9defcdd924427f571a0518b7cc3b2244c0a246fb6914931e5ea3bb561eac8b"} Jul 30 06:51:07.668606 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:07.668319 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc4596cdf-vt55p" event={"ID":"b3e757c4-bb98-457a-9992-5b738f8a8249","Type":"ContainerDied","Data":"576114c7059613d7a5fb3bdcc11f73af151c60df6f8bc8755e054c7cf9227299"} Jul 30 06:51:07.668606 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:07.668340 2575 scope.go:117] "RemoveContainer" containerID="5e9defcdd924427f571a0518b7cc3b2244c0a246fb6914931e5ea3bb561eac8b" Jul 30 06:51:07.677001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:07.676986 2575 scope.go:117] "RemoveContainer" containerID="5e9defcdd924427f571a0518b7cc3b2244c0a246fb6914931e5ea3bb561eac8b" Jul 30 06:51:07.677273 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:51:07.677253 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e9defcdd924427f571a0518b7cc3b2244c0a246fb6914931e5ea3bb561eac8b\": container with ID starting with 5e9defcdd924427f571a0518b7cc3b2244c0a246fb6914931e5ea3bb561eac8b not found: ID does not exist" containerID="5e9defcdd924427f571a0518b7cc3b2244c0a246fb6914931e5ea3bb561eac8b" Jul 30 06:51:07.677324 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:07.677283 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e9defcdd924427f571a0518b7cc3b2244c0a246fb6914931e5ea3bb561eac8b"} err="failed to get container status \"5e9defcdd924427f571a0518b7cc3b2244c0a246fb6914931e5ea3bb561eac8b\": rpc error: code = NotFound desc = could not find container \"5e9defcdd924427f571a0518b7cc3b2244c0a246fb6914931e5ea3bb561eac8b\": container with ID starting with 5e9defcdd924427f571a0518b7cc3b2244c0a246fb6914931e5ea3bb561eac8b not found: ID does not exist" Jul 30 06:51:07.685123 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:07.685101 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cc4596cdf-vt55p"] Jul 30 06:51:07.690426 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:07.690407 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cc4596cdf-vt55p"] Jul 30 06:51:09.180729 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:09.180694 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3e757c4-bb98-457a-9992-5b738f8a8249" path="/var/lib/kubelet/pods/b3e757c4-bb98-457a-9992-5b738f8a8249/volumes" Jul 30 06:51:36.831300 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.831246 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d5864f6db-d22nz"] Jul 30 06:51:36.831772 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.831551 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3e757c4-bb98-457a-9992-5b738f8a8249" containerName="console" Jul 30 06:51:36.831772 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.831564 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3e757c4-bb98-457a-9992-5b738f8a8249" containerName="console" Jul 30 06:51:36.831772 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.831672 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3e757c4-bb98-457a-9992-5b738f8a8249" containerName="console" Jul 30 06:51:36.834768 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.834749 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.845553 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.845527 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d5864f6db-d22nz"] Jul 30 06:51:36.857387 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.857356 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl8ws\" (UniqueName: \"kubernetes.io/projected/31175bf8-8013-4336-9be6-0bc787b46884-kube-api-access-sl8ws\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.857525 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.857403 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31175bf8-8013-4336-9be6-0bc787b46884-console-oauth-config\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.857525 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.857440 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-oauth-serving-cert\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.857525 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.857465 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-service-ca\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.857795 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.857773 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31175bf8-8013-4336-9be6-0bc787b46884-console-serving-cert\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.857852 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.857809 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-trusted-ca-bundle\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.857923 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.857906 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-console-config\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.958438 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.958393 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl8ws\" (UniqueName: \"kubernetes.io/projected/31175bf8-8013-4336-9be6-0bc787b46884-kube-api-access-sl8ws\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.958438 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.958441 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31175bf8-8013-4336-9be6-0bc787b46884-console-oauth-config\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.958713 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.958469 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-oauth-serving-cert\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.958713 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.958486 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-service-ca\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.958713 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.958506 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31175bf8-8013-4336-9be6-0bc787b46884-console-serving-cert\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.958713 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.958520 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-trusted-ca-bundle\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.958713 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.958552 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-console-config\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.959363 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.959335 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-oauth-serving-cert\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.959484 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.959347 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-console-config\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.959522 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.959502 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-service-ca\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.959672 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.959636 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-trusted-ca-bundle\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.961123 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.961104 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31175bf8-8013-4336-9be6-0bc787b46884-console-serving-cert\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.961194 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.961126 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31175bf8-8013-4336-9be6-0bc787b46884-console-oauth-config\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:36.965856 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:36.965837 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl8ws\" (UniqueName: \"kubernetes.io/projected/31175bf8-8013-4336-9be6-0bc787b46884-kube-api-access-sl8ws\") pod \"console-5d5864f6db-d22nz\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:37.145282 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:37.145251 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:37.278173 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:37.278039 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d5864f6db-d22nz"] Jul 30 06:51:37.280983 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:51:37.280956 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31175bf8_8013_4336_9be6_0bc787b46884.slice/crio-b06b5b94f3420215249a5f84cb374c5a79322cbef03de17e5e55a0def12dacdd WatchSource:0}: Error finding container b06b5b94f3420215249a5f84cb374c5a79322cbef03de17e5e55a0def12dacdd: Status 404 returned error can't find the container with id b06b5b94f3420215249a5f84cb374c5a79322cbef03de17e5e55a0def12dacdd Jul 30 06:51:37.785563 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:37.785531 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5864f6db-d22nz" event={"ID":"31175bf8-8013-4336-9be6-0bc787b46884","Type":"ContainerStarted","Data":"a71e4f4826f00a2a6734f8e9293e967bd54020faf71a2551abf5a1517a480cc0"} Jul 30 06:51:37.785784 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:37.785577 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5864f6db-d22nz" event={"ID":"31175bf8-8013-4336-9be6-0bc787b46884","Type":"ContainerStarted","Data":"b06b5b94f3420215249a5f84cb374c5a79322cbef03de17e5e55a0def12dacdd"} Jul 30 06:51:37.803676 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:37.803618 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d5864f6db-d22nz" podStartSLOduration=1.803600106 podStartE2EDuration="1.803600106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:51:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:51:37.802578354 +0000 UTC m=+459.159582515" watchObservedRunningTime="2026-07-30 06:51:37.803600106 +0000 UTC m=+459.160604268" Jul 30 06:51:47.145800 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:47.145760 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:47.146207 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:47.145832 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:47.150781 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:47.150753 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:47.826918 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:47.826891 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:51:47.869948 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:51:47.869912 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d447b797b-h2vgf"] Jul 30 06:52:12.889799 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:12.889740 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d447b797b-h2vgf" podUID="4698337d-9c90-4e8b-90a0-c2ad585d1748" containerName="console" containerID="cri-o://fbb0bb9d843cf113e48114b74fe9c533f357af2228f7b3250b319b8ff3eda52d" gracePeriod=15 Jul 30 06:52:13.131019 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.130993 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:52:13.249018 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.248937 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-service-ca\") pod \"4698337d-9c90-4e8b-90a0-c2ad585d1748\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " Jul 30 06:52:13.249018 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.248969 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-oauth-serving-cert\") pod \"4698337d-9c90-4e8b-90a0-c2ad585d1748\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " Jul 30 06:52:13.249233 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.249024 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-config\") pod \"4698337d-9c90-4e8b-90a0-c2ad585d1748\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " Jul 30 06:52:13.249233 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.249061 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-oauth-config\") pod \"4698337d-9c90-4e8b-90a0-c2ad585d1748\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " Jul 30 06:52:13.249233 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.249082 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6c8sk\" (UniqueName: \"kubernetes.io/projected/4698337d-9c90-4e8b-90a0-c2ad585d1748-kube-api-access-6c8sk\") pod \"4698337d-9c90-4e8b-90a0-c2ad585d1748\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " Jul 30 06:52:13.249233 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.249110 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-serving-cert\") pod \"4698337d-9c90-4e8b-90a0-c2ad585d1748\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " Jul 30 06:52:13.249233 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.249149 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-trusted-ca-bundle\") pod \"4698337d-9c90-4e8b-90a0-c2ad585d1748\" (UID: \"4698337d-9c90-4e8b-90a0-c2ad585d1748\") " Jul 30 06:52:13.249484 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.249420 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-service-ca" (OuterVolumeSpecName: "service-ca") pod "4698337d-9c90-4e8b-90a0-c2ad585d1748" (UID: "4698337d-9c90-4e8b-90a0-c2ad585d1748"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:52:13.249537 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.249474 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-config" (OuterVolumeSpecName: "console-config") pod "4698337d-9c90-4e8b-90a0-c2ad585d1748" (UID: "4698337d-9c90-4e8b-90a0-c2ad585d1748"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:52:13.249537 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.249482 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4698337d-9c90-4e8b-90a0-c2ad585d1748" (UID: "4698337d-9c90-4e8b-90a0-c2ad585d1748"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:52:13.250083 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.250045 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4698337d-9c90-4e8b-90a0-c2ad585d1748" (UID: "4698337d-9c90-4e8b-90a0-c2ad585d1748"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:52:13.251415 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.251392 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4698337d-9c90-4e8b-90a0-c2ad585d1748" (UID: "4698337d-9c90-4e8b-90a0-c2ad585d1748"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:52:13.251868 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.251851 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4698337d-9c90-4e8b-90a0-c2ad585d1748-kube-api-access-6c8sk" (OuterVolumeSpecName: "kube-api-access-6c8sk") pod "4698337d-9c90-4e8b-90a0-c2ad585d1748" (UID: "4698337d-9c90-4e8b-90a0-c2ad585d1748"). InnerVolumeSpecName "kube-api-access-6c8sk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 06:52:13.251868 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.251860 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4698337d-9c90-4e8b-90a0-c2ad585d1748" (UID: "4698337d-9c90-4e8b-90a0-c2ad585d1748"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:52:13.350369 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.350335 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-serving-cert\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:52:13.350369 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.350361 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-trusted-ca-bundle\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:52:13.350369 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.350371 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-service-ca\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:52:13.350369 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.350379 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-oauth-serving-cert\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:52:13.350617 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.350388 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-config\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:52:13.350617 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.350397 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4698337d-9c90-4e8b-90a0-c2ad585d1748-console-oauth-config\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:52:13.350617 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.350407 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6c8sk\" (UniqueName: \"kubernetes.io/projected/4698337d-9c90-4e8b-90a0-c2ad585d1748-kube-api-access-6c8sk\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:52:13.925546 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.925514 2575 generic.go:356] "Generic (PLEG): container finished" podID="4698337d-9c90-4e8b-90a0-c2ad585d1748" containerID="fbb0bb9d843cf113e48114b74fe9c533f357af2228f7b3250b319b8ff3eda52d" exitCode=0 Jul 30 06:52:13.925955 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.925554 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d447b797b-h2vgf" Jul 30 06:52:13.925955 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.925592 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d447b797b-h2vgf" event={"ID":"4698337d-9c90-4e8b-90a0-c2ad585d1748","Type":"ContainerDied","Data":"fbb0bb9d843cf113e48114b74fe9c533f357af2228f7b3250b319b8ff3eda52d"} Jul 30 06:52:13.925955 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.925643 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d447b797b-h2vgf" event={"ID":"4698337d-9c90-4e8b-90a0-c2ad585d1748","Type":"ContainerDied","Data":"d6032c5638a8f353ca9679ce5a3ec6fb4c6b10400d930858d76e4e6137e13f85"} Jul 30 06:52:13.925955 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.925686 2575 scope.go:117] "RemoveContainer" containerID="fbb0bb9d843cf113e48114b74fe9c533f357af2228f7b3250b319b8ff3eda52d" Jul 30 06:52:13.934484 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.934469 2575 scope.go:117] "RemoveContainer" containerID="fbb0bb9d843cf113e48114b74fe9c533f357af2228f7b3250b319b8ff3eda52d" Jul 30 06:52:13.934771 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:52:13.934753 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbb0bb9d843cf113e48114b74fe9c533f357af2228f7b3250b319b8ff3eda52d\": container with ID starting with fbb0bb9d843cf113e48114b74fe9c533f357af2228f7b3250b319b8ff3eda52d not found: ID does not exist" containerID="fbb0bb9d843cf113e48114b74fe9c533f357af2228f7b3250b319b8ff3eda52d" Jul 30 06:52:13.934848 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.934784 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbb0bb9d843cf113e48114b74fe9c533f357af2228f7b3250b319b8ff3eda52d"} err="failed to get container status \"fbb0bb9d843cf113e48114b74fe9c533f357af2228f7b3250b319b8ff3eda52d\": rpc error: code = NotFound desc = could not find container \"fbb0bb9d843cf113e48114b74fe9c533f357af2228f7b3250b319b8ff3eda52d\": container with ID starting with fbb0bb9d843cf113e48114b74fe9c533f357af2228f7b3250b319b8ff3eda52d not found: ID does not exist" Jul 30 06:52:13.945393 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.945366 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d447b797b-h2vgf"] Jul 30 06:52:13.950251 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:13.950221 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d447b797b-h2vgf"] Jul 30 06:52:15.175608 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:52:15.175575 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4698337d-9c90-4e8b-90a0-c2ad585d1748" path="/var/lib/kubelet/pods/4698337d-9c90-4e8b-90a0-c2ad585d1748/volumes" Jul 30 06:53:31.998831 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:31.998794 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4"] Jul 30 06:53:31.999246 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:31.999100 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4698337d-9c90-4e8b-90a0-c2ad585d1748" containerName="console" Jul 30 06:53:31.999246 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:31.999113 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4698337d-9c90-4e8b-90a0-c2ad585d1748" containerName="console" Jul 30 06:53:31.999246 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:31.999168 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4698337d-9c90-4e8b-90a0-c2ad585d1748" containerName="console" Jul 30 06:53:32.002021 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:32.002001 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4" Jul 30 06:53:32.004718 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:32.004691 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Jul 30 06:53:32.004837 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:32.004725 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Jul 30 06:53:32.004837 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:32.004734 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-f4xwl\"" Jul 30 06:53:32.010797 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:32.010777 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4"] Jul 30 06:53:32.098405 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:32.098362 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdf7l\" (UniqueName: \"kubernetes.io/projected/cfc51ead-a2f4-465a-aa9d-7c3f075c06e2-kube-api-access-tdf7l\") pod \"feast-operator-controller-manager-7884ccc455-vxvq4\" (UID: \"cfc51ead-a2f4-465a-aa9d-7c3f075c06e2\") " pod="feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4" Jul 30 06:53:32.198791 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:32.198762 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdf7l\" (UniqueName: \"kubernetes.io/projected/cfc51ead-a2f4-465a-aa9d-7c3f075c06e2-kube-api-access-tdf7l\") pod \"feast-operator-controller-manager-7884ccc455-vxvq4\" (UID: \"cfc51ead-a2f4-465a-aa9d-7c3f075c06e2\") " pod="feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4" Jul 30 06:53:32.207298 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:32.207278 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdf7l\" (UniqueName: \"kubernetes.io/projected/cfc51ead-a2f4-465a-aa9d-7c3f075c06e2-kube-api-access-tdf7l\") pod \"feast-operator-controller-manager-7884ccc455-vxvq4\" (UID: \"cfc51ead-a2f4-465a-aa9d-7c3f075c06e2\") " pod="feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4" Jul 30 06:53:32.311985 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:32.311899 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4" Jul 30 06:53:32.444669 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:32.444627 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4"] Jul 30 06:53:32.447071 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:53:32.447038 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfc51ead_a2f4_465a_aa9d_7c3f075c06e2.slice/crio-9c806b96ef8d9631b23ea1bd3d715ebf3167422cd9119dd74725d2cd0b03a89b WatchSource:0}: Error finding container 9c806b96ef8d9631b23ea1bd3d715ebf3167422cd9119dd74725d2cd0b03a89b: Status 404 returned error can't find the container with id 9c806b96ef8d9631b23ea1bd3d715ebf3167422cd9119dd74725d2cd0b03a89b Jul 30 06:53:33.225483 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:33.225440 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4" event={"ID":"cfc51ead-a2f4-465a-aa9d-7c3f075c06e2","Type":"ContainerStarted","Data":"9c806b96ef8d9631b23ea1bd3d715ebf3167422cd9119dd74725d2cd0b03a89b"} Jul 30 06:53:35.235515 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:35.235479 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4" event={"ID":"cfc51ead-a2f4-465a-aa9d-7c3f075c06e2","Type":"ContainerStarted","Data":"2ccfe685ecde79f9dde6e87c821a9e96613b40d1374672df9271a5358fc7361d"} Jul 30 06:53:35.235973 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:35.235622 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4" Jul 30 06:53:35.251598 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:35.251552 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4" podStartSLOduration=1.979397791 podStartE2EDuration="4.251537757s" totalImagesPullingTime="2.272139966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:53:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:53:35.249866182 +0000 UTC m=+576.606870344" watchObservedRunningTime="2026-07-30 06:53:35.251537757 +0000 UTC m=+576.608541919" Jul 30 06:53:46.242242 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:46.242209 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4" Jul 30 06:53:59.235815 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:59.235715 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/1.log" Jul 30 06:53:59.238877 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:59.236678 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/1.log" Jul 30 06:53:59.242651 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:59.242631 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 06:53:59.243074 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:53:59.243057 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 06:54:09.050437 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.050403 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr"] Jul 30 06:54:09.057580 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.057562 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.060342 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.060313 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 30 06:54:09.060342 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.060337 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jul 30 06:54:09.060586 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.060390 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 30 06:54:09.060586 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.060426 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jul 30 06:54:09.060586 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.060392 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jul 30 06:54:09.060586 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.060511 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-h8qg5\"" Jul 30 06:54:09.060768 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.060651 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jul 30 06:54:09.060810 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.060803 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Jul 30 06:54:09.064961 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.064936 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr"] Jul 30 06:54:09.214843 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.214813 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-offline-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.214843 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.214846 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-ui-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.215075 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.214872 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-registry-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.215075 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.214982 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-online-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.215075 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.215008 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x7nz\" (UniqueName: \"kubernetes.io/projected/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-kube-api-access-9x7nz\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.215075 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.215040 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-feast-data\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.315914 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.315832 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-offline-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.315914 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.315864 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-ui-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.315914 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.315881 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-registry-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.316150 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:54:09.315987 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Jul 30 06:54:09.316150 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.316021 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-online-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.316150 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:54:09.316056 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-registry-tls podName:ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7 nodeName:}" failed. No retries permitted until 2026-07-30 06:54:09.816036171 +0000 UTC m=+611.173040334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-registry-tls") pod "feast-simple-feast-setup-7b7885d7d4-kfkdr" (UID: "ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7") : secret "feast-simple-feast-setup-registry-tls" not found Jul 30 06:54:09.316150 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.316074 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x7nz\" (UniqueName: \"kubernetes.io/projected/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-kube-api-access-9x7nz\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.316150 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.316115 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-feast-data\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.316475 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.316456 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-feast-data\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.318479 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.318460 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-offline-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.318604 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.318509 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-ui-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.318604 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.318540 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-online-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.327168 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.327144 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x7nz\" (UniqueName: \"kubernetes.io/projected/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-kube-api-access-9x7nz\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.820591 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.820554 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-registry-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.823137 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.823114 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-registry-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-kfkdr\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:09.978758 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:09.978718 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:10.120456 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:10.120428 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr"] Jul 30 06:54:10.121826 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:54:10.121800 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab9fc9d7_fbae_472a_99ef_0a7c27fa65a7.slice/crio-d977b8a5b2d5a5989d42914b5888fe4e6335633c1c1f063dce9a2ae987a1e7ee WatchSource:0}: Error finding container d977b8a5b2d5a5989d42914b5888fe4e6335633c1c1f063dce9a2ae987a1e7ee: Status 404 returned error can't find the container with id d977b8a5b2d5a5989d42914b5888fe4e6335633c1c1f063dce9a2ae987a1e7ee Jul 30 06:54:10.123802 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:10.123786 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 06:54:10.382602 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:10.382569 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" event={"ID":"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7","Type":"ContainerStarted","Data":"d977b8a5b2d5a5989d42914b5888fe4e6335633c1c1f063dce9a2ae987a1e7ee"} Jul 30 06:54:35.498334 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:35.497407 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" event={"ID":"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7","Type":"ContainerStarted","Data":"f3f4de490cd934747e13af87d1e8d384822f1830b917f97d85c4d01ac5f10627"} Jul 30 06:54:40.517993 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:40.517913 2575 generic.go:356] "Generic (PLEG): container finished" podID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerID="f3f4de490cd934747e13af87d1e8d384822f1830b917f97d85c4d01ac5f10627" exitCode=0 Jul 30 06:54:40.517993 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:40.517980 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" event={"ID":"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7","Type":"ContainerDied","Data":"f3f4de490cd934747e13af87d1e8d384822f1830b917f97d85c4d01ac5f10627"} Jul 30 06:54:41.524825 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:41.524785 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" event={"ID":"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7","Type":"ContainerStarted","Data":"fa0878cf65ff0c1e54f0ab1c2712ea86259bbf87fe1575cebb0688fd7d22d5d5"} Jul 30 06:54:46.546827 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:46.546741 2575 generic.go:356] "Generic (PLEG): container finished" podID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerID="fa0878cf65ff0c1e54f0ab1c2712ea86259bbf87fe1575cebb0688fd7d22d5d5" exitCode=0 Jul 30 06:54:46.546827 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:46.546788 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" event={"ID":"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7","Type":"ContainerDied","Data":"fa0878cf65ff0c1e54f0ab1c2712ea86259bbf87fe1575cebb0688fd7d22d5d5"} Jul 30 06:54:47.556813 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:47.556765 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" event={"ID":"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7","Type":"ContainerStarted","Data":"d2c5ed9c186d63d9b32c127665f4893b06b621e61cf8fe6b1fd13b05e96b59c7"} Jul 30 06:54:47.557271 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:47.556831 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" event={"ID":"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7","Type":"ContainerStarted","Data":"6f8eefc72825f8bd73663d3f60ddffffb024a26204342f68c2d065961707a544"} Jul 30 06:54:47.557271 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:47.556854 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" event={"ID":"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7","Type":"ContainerStarted","Data":"0b0e902bfb80c07fb51a2d3353e39e77aeb891414e6f404798c9f36273242488"} Jul 30 06:54:47.557271 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:47.556874 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" event={"ID":"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7","Type":"ContainerStarted","Data":"d6423ec9a4d241ca997dac940246781ff51f5686412ccf46b7f6e4aa6e71f81c"} Jul 30 06:54:47.583549 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:47.583467 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podStartSLOduration=13.464758029 podStartE2EDuration="38.583444162s" totalImagesPullingTime="25.118686133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:54:47.580161505 +0000 UTC m=+648.937165669" watchObservedRunningTime="2026-07-30 06:54:47.583444162 +0000 UTC m=+648.940448325" Jul 30 06:54:48.979254 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:48.979120 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:48.979254 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:48.979185 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:48.979254 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:48.979205 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:48.979254 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:48.979223 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:48.981266 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:48.981231 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused" Jul 30 06:54:48.981678 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:48.981252 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.26:6571: connect: connection refused" Jul 30 06:54:48.981802 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:48.981262 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.26:8816: connect: connection refused" Jul 30 06:54:48.981802 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:48.981338 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Jul 30 06:54:49.979565 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:49.979502 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:49.980102 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:49.979578 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:49.980102 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:49.979602 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:49.980102 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:49.979620 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:51.979539 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:51.979495 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.26:6571: connect: connection refused" Jul 30 06:54:51.980238 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:51.979497 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused" Jul 30 06:54:51.980321 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:51.979647 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Jul 30 06:54:51.980321 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:51.979581 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.26:8816: connect: connection refused" Jul 30 06:54:54.980497 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:54.980468 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:54.980925 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:54.980621 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:54.980925 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:54.980901 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:54.981461 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:54.981419 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:54.981564 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:54.981501 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:54.981564 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:54.981550 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:54.985693 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:54.985672 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:54:55.600010 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:54:55.599980 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:55:43.801123 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:43.801086 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d5864f6db-d22nz"] Jul 30 06:55:58.404122 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.404038 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr"] Jul 30 06:55:58.404740 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.404325 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="offline" containerID="cri-o://6f8eefc72825f8bd73663d3f60ddffffb024a26204342f68c2d065961707a544" gracePeriod=30 Jul 30 06:55:58.404740 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.404339 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="online" containerID="cri-o://0b0e902bfb80c07fb51a2d3353e39e77aeb891414e6f404798c9f36273242488" gracePeriod=30 Jul 30 06:55:58.404740 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.404337 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="ui" containerID="cri-o://d2c5ed9c186d63d9b32c127665f4893b06b621e61cf8fe6b1fd13b05e96b59c7" gracePeriod=30 Jul 30 06:55:58.404740 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.404486 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="registry" containerID="cri-o://d6423ec9a4d241ca997dac940246781ff51f5686412ccf46b7f6e4aa6e71f81c" gracePeriod=30 Jul 30 06:55:58.676046 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.675955 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs"] Jul 30 06:55:58.679045 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.679020 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.682454 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.682432 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-2dpbp\"" Jul 30 06:55:58.695846 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.695811 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs"] Jul 30 06:55:58.777640 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.777603 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-ui-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.777853 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.777643 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-offline-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.777853 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.777716 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbqrw\" (UniqueName: \"kubernetes.io/projected/142df858-460c-4651-ba54-577c0f134c2a-kube-api-access-jbqrw\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.777853 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.777735 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/142df858-460c-4651-ba54-577c0f134c2a-feast-data\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.777853 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.777792 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-online-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.778061 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.777915 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-registry-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.879096 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.879047 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-registry-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.879096 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.879094 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-ui-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.879375 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.879114 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-offline-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.879375 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.879142 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbqrw\" (UniqueName: \"kubernetes.io/projected/142df858-460c-4651-ba54-577c0f134c2a-kube-api-access-jbqrw\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.879375 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.879171 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/142df858-460c-4651-ba54-577c0f134c2a-feast-data\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.879375 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.879200 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-online-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.879375 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:55:58.879229 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Jul 30 06:55:58.879375 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:55:58.879304 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-registry-tls podName:142df858-460c-4651-ba54-577c0f134c2a nodeName:}" failed. No retries permitted until 2026-07-30 06:55:59.379281728 +0000 UTC m=+720.736285893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-registry-tls") pod "feast-simple-feast-setup-7b7885d7d4-x5sbs" (UID: "142df858-460c-4651-ba54-577c0f134c2a") : secret "feast-simple-feast-setup-registry-tls" not found Jul 30 06:55:58.879731 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.879715 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/142df858-460c-4651-ba54-577c0f134c2a-feast-data\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.881886 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.881857 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-ui-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.882025 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.881993 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-online-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.882092 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.882072 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-offline-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:58.888182 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:58.888157 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbqrw\" (UniqueName: \"kubernetes.io/projected/142df858-460c-4651-ba54-577c0f134c2a-kube-api-access-jbqrw\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:59.382858 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:59.382828 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-registry-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:59.385376 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:59.385350 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-registry-tls\") pod \"feast-simple-feast-setup-7b7885d7d4-x5sbs\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:59.592630 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:59.592600 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-2dpbp\"" Jul 30 06:55:59.599644 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:59.599625 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:55:59.757108 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:59.757078 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs"] Jul 30 06:55:59.759432 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:55:59.759405 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod142df858_460c_4651_ba54_577c0f134c2a.slice/crio-87dae8d4219665c3b46f59979fe27cacd1773ed6d9351c1d29db7d85fb7ec052 WatchSource:0}: Error finding container 87dae8d4219665c3b46f59979fe27cacd1773ed6d9351c1d29db7d85fb7ec052: Status 404 returned error can't find the container with id 87dae8d4219665c3b46f59979fe27cacd1773ed6d9351c1d29db7d85fb7ec052 Jul 30 06:55:59.853938 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:59.853910 2575 generic.go:356] "Generic (PLEG): container finished" podID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerID="d2c5ed9c186d63d9b32c127665f4893b06b621e61cf8fe6b1fd13b05e96b59c7" exitCode=0 Jul 30 06:55:59.853938 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:59.853935 2575 generic.go:356] "Generic (PLEG): container finished" podID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerID="0b0e902bfb80c07fb51a2d3353e39e77aeb891414e6f404798c9f36273242488" exitCode=0 Jul 30 06:55:59.854099 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:59.853976 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" event={"ID":"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7","Type":"ContainerDied","Data":"d2c5ed9c186d63d9b32c127665f4893b06b621e61cf8fe6b1fd13b05e96b59c7"} Jul 30 06:55:59.854099 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:59.854017 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" event={"ID":"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7","Type":"ContainerDied","Data":"0b0e902bfb80c07fb51a2d3353e39e77aeb891414e6f404798c9f36273242488"} Jul 30 06:55:59.855454 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:59.855430 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" event={"ID":"142df858-460c-4651-ba54-577c0f134c2a","Type":"ContainerStarted","Data":"41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe"} Jul 30 06:55:59.855709 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:55:59.855463 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" event={"ID":"142df858-460c-4651-ba54-577c0f134c2a","Type":"ContainerStarted","Data":"87dae8d4219665c3b46f59979fe27cacd1773ed6d9351c1d29db7d85fb7ec052"} Jul 30 06:56:04.981626 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:04.981577 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused" Jul 30 06:56:05.597302 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:05.597253 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Jul 30 06:56:05.881328 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:05.881291 2575 generic.go:356] "Generic (PLEG): container finished" podID="142df858-460c-4651-ba54-577c0f134c2a" containerID="41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe" exitCode=0 Jul 30 06:56:05.881455 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:05.881365 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" event={"ID":"142df858-460c-4651-ba54-577c0f134c2a","Type":"ContainerDied","Data":"41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe"} Jul 30 06:56:06.888495 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:06.888460 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" event={"ID":"142df858-460c-4651-ba54-577c0f134c2a","Type":"ContainerStarted","Data":"ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490"} Jul 30 06:56:08.821170 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:08.821122 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d5864f6db-d22nz" podUID="31175bf8-8013-4336-9be6-0bc787b46884" containerName="console" containerID="cri-o://a71e4f4826f00a2a6734f8e9293e967bd54020faf71a2551abf5a1517a480cc0" gracePeriod=15 Jul 30 06:56:09.075183 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.075124 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:56:09.172329 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.172292 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31175bf8-8013-4336-9be6-0bc787b46884-console-serving-cert\") pod \"31175bf8-8013-4336-9be6-0bc787b46884\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " Jul 30 06:56:09.172497 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.172360 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31175bf8-8013-4336-9be6-0bc787b46884-console-oauth-config\") pod \"31175bf8-8013-4336-9be6-0bc787b46884\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " Jul 30 06:56:09.172497 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.172424 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-console-config\") pod \"31175bf8-8013-4336-9be6-0bc787b46884\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " Jul 30 06:56:09.172497 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.172472 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-service-ca\") pod \"31175bf8-8013-4336-9be6-0bc787b46884\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " Jul 30 06:56:09.172645 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.172509 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-trusted-ca-bundle\") pod \"31175bf8-8013-4336-9be6-0bc787b46884\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " Jul 30 06:56:09.172645 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.172541 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sl8ws\" (UniqueName: \"kubernetes.io/projected/31175bf8-8013-4336-9be6-0bc787b46884-kube-api-access-sl8ws\") pod \"31175bf8-8013-4336-9be6-0bc787b46884\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " Jul 30 06:56:09.172645 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.172564 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-oauth-serving-cert\") pod \"31175bf8-8013-4336-9be6-0bc787b46884\" (UID: \"31175bf8-8013-4336-9be6-0bc787b46884\") " Jul 30 06:56:09.173354 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.173040 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-console-config" (OuterVolumeSpecName: "console-config") pod "31175bf8-8013-4336-9be6-0bc787b46884" (UID: "31175bf8-8013-4336-9be6-0bc787b46884"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:56:09.173354 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.173156 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "31175bf8-8013-4336-9be6-0bc787b46884" (UID: "31175bf8-8013-4336-9be6-0bc787b46884"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:56:09.173354 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.173216 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-service-ca" (OuterVolumeSpecName: "service-ca") pod "31175bf8-8013-4336-9be6-0bc787b46884" (UID: "31175bf8-8013-4336-9be6-0bc787b46884"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:56:09.173572 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.173388 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "31175bf8-8013-4336-9be6-0bc787b46884" (UID: "31175bf8-8013-4336-9be6-0bc787b46884"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:56:09.174996 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.174971 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31175bf8-8013-4336-9be6-0bc787b46884-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "31175bf8-8013-4336-9be6-0bc787b46884" (UID: "31175bf8-8013-4336-9be6-0bc787b46884"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:56:09.174996 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.174988 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31175bf8-8013-4336-9be6-0bc787b46884-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "31175bf8-8013-4336-9be6-0bc787b46884" (UID: "31175bf8-8013-4336-9be6-0bc787b46884"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:56:09.175515 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.175493 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31175bf8-8013-4336-9be6-0bc787b46884-kube-api-access-sl8ws" (OuterVolumeSpecName: "kube-api-access-sl8ws") pod "31175bf8-8013-4336-9be6-0bc787b46884" (UID: "31175bf8-8013-4336-9be6-0bc787b46884"). InnerVolumeSpecName "kube-api-access-sl8ws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 06:56:09.273449 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.273417 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31175bf8-8013-4336-9be6-0bc787b46884-console-oauth-config\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:56:09.273449 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.273443 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-console-config\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:56:09.273449 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.273454 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-service-ca\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:56:09.273730 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.273462 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-trusted-ca-bundle\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:56:09.273730 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.273471 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sl8ws\" (UniqueName: \"kubernetes.io/projected/31175bf8-8013-4336-9be6-0bc787b46884-kube-api-access-sl8ws\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:56:09.273730 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.273480 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31175bf8-8013-4336-9be6-0bc787b46884-oauth-serving-cert\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:56:09.273730 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.273488 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31175bf8-8013-4336-9be6-0bc787b46884-console-serving-cert\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:56:09.903249 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.903218 2575 generic.go:356] "Generic (PLEG): container finished" podID="31175bf8-8013-4336-9be6-0bc787b46884" containerID="a71e4f4826f00a2a6734f8e9293e967bd54020faf71a2551abf5a1517a480cc0" exitCode=0 Jul 30 06:56:09.903703 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.903250 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5864f6db-d22nz" event={"ID":"31175bf8-8013-4336-9be6-0bc787b46884","Type":"ContainerDied","Data":"a71e4f4826f00a2a6734f8e9293e967bd54020faf71a2551abf5a1517a480cc0"} Jul 30 06:56:09.903703 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.903288 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5864f6db-d22nz" Jul 30 06:56:09.903703 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.903318 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5864f6db-d22nz" event={"ID":"31175bf8-8013-4336-9be6-0bc787b46884","Type":"ContainerDied","Data":"b06b5b94f3420215249a5f84cb374c5a79322cbef03de17e5e55a0def12dacdd"} Jul 30 06:56:09.903703 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.903341 2575 scope.go:117] "RemoveContainer" containerID="a71e4f4826f00a2a6734f8e9293e967bd54020faf71a2551abf5a1517a480cc0" Jul 30 06:56:09.920241 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.920221 2575 scope.go:117] "RemoveContainer" containerID="a71e4f4826f00a2a6734f8e9293e967bd54020faf71a2551abf5a1517a480cc0" Jul 30 06:56:09.920496 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:56:09.920478 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a71e4f4826f00a2a6734f8e9293e967bd54020faf71a2551abf5a1517a480cc0\": container with ID starting with a71e4f4826f00a2a6734f8e9293e967bd54020faf71a2551abf5a1517a480cc0 not found: ID does not exist" containerID="a71e4f4826f00a2a6734f8e9293e967bd54020faf71a2551abf5a1517a480cc0" Jul 30 06:56:09.920560 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.920507 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a71e4f4826f00a2a6734f8e9293e967bd54020faf71a2551abf5a1517a480cc0"} err="failed to get container status \"a71e4f4826f00a2a6734f8e9293e967bd54020faf71a2551abf5a1517a480cc0\": rpc error: code = NotFound desc = could not find container \"a71e4f4826f00a2a6734f8e9293e967bd54020faf71a2551abf5a1517a480cc0\": container with ID starting with a71e4f4826f00a2a6734f8e9293e967bd54020faf71a2551abf5a1517a480cc0 not found: ID does not exist" Jul 30 06:56:09.925776 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.925753 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d5864f6db-d22nz"] Jul 30 06:56:09.929448 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:09.929427 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d5864f6db-d22nz"] Jul 30 06:56:11.176178 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:11.176142 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31175bf8-8013-4336-9be6-0bc787b46884" path="/var/lib/kubelet/pods/31175bf8-8013-4336-9be6-0bc787b46884/volumes" Jul 30 06:56:11.915103 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:11.915069 2575 generic.go:356] "Generic (PLEG): container finished" podID="142df858-460c-4651-ba54-577c0f134c2a" containerID="ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490" exitCode=0 Jul 30 06:56:11.915291 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:11.915142 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" event={"ID":"142df858-460c-4651-ba54-577c0f134c2a","Type":"ContainerDied","Data":"ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490"} Jul 30 06:56:12.925011 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:12.924968 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" event={"ID":"142df858-460c-4651-ba54-577c0f134c2a","Type":"ContainerStarted","Data":"78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462"} Jul 30 06:56:12.925491 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:12.925038 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" event={"ID":"142df858-460c-4651-ba54-577c0f134c2a","Type":"ContainerStarted","Data":"4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3"} Jul 30 06:56:12.925491 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:12.925059 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" event={"ID":"142df858-460c-4651-ba54-577c0f134c2a","Type":"ContainerStarted","Data":"b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1"} Jul 30 06:56:12.925491 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:12.925077 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" event={"ID":"142df858-460c-4651-ba54-577c0f134c2a","Type":"ContainerStarted","Data":"605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c"} Jul 30 06:56:12.952875 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:12.952811 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podStartSLOduration=14.952785192 podStartE2EDuration="14.952785192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:55:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:56:12.952378099 +0000 UTC m=+734.309382273" watchObservedRunningTime="2026-07-30 06:56:12.952785192 +0000 UTC m=+734.309789355" Jul 30 06:56:14.600262 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:14.600218 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:56:14.600757 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:14.600283 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:56:14.600757 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:14.600306 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:56:14.600757 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:14.600324 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:56:14.602227 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:14.602193 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.27:6571: connect: connection refused" Jul 30 06:56:14.602358 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:14.602226 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.27:8816: connect: connection refused" Jul 30 06:56:14.602358 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:14.602340 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jul 30 06:56:14.602483 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:14.602362 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Jul 30 06:56:14.981451 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:14.981406 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused" Jul 30 06:56:15.597284 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:15.597231 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Jul 30 06:56:17.600432 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:17.600381 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.27:8816: connect: connection refused" Jul 30 06:56:17.600937 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:17.600388 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.27:6571: connect: connection refused" Jul 30 06:56:17.600937 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:17.600494 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Jul 30 06:56:17.601143 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:17.601101 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jul 30 06:56:19.600033 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:19.599982 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:56:19.600033 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:19.600056 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:56:19.600485 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:19.600071 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:56:19.600485 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:19.600082 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:56:20.601552 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:20.601473 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:56:20.602005 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:20.601609 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:56:20.602005 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:20.601954 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:56:20.602591 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:20.602567 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:56:20.602742 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:20.602634 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:56:20.602742 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:20.602699 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:56:20.606402 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:20.606384 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:56:20.973318 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:20.973291 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:56:24.982177 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:24.982113 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.26:8443: connect: connection refused" Jul 30 06:56:24.982619 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:24.982283 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:56:25.597101 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:25.597061 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Jul 30 06:56:25.597268 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:25.597172 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:56:29.007157 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.007126 2575 generic.go:356] "Generic (PLEG): container finished" podID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerID="6f8eefc72825f8bd73663d3f60ddffffb024a26204342f68c2d065961707a544" exitCode=137 Jul 30 06:56:29.007157 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.007151 2575 generic.go:356] "Generic (PLEG): container finished" podID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerID="d6423ec9a4d241ca997dac940246781ff51f5686412ccf46b7f6e4aa6e71f81c" exitCode=137 Jul 30 06:56:29.007683 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.007174 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" event={"ID":"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7","Type":"ContainerDied","Data":"6f8eefc72825f8bd73663d3f60ddffffb024a26204342f68c2d065961707a544"} Jul 30 06:56:29.007683 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.007218 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" event={"ID":"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7","Type":"ContainerDied","Data":"d6423ec9a4d241ca997dac940246781ff51f5686412ccf46b7f6e4aa6e71f81c"} Jul 30 06:56:29.056451 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.056429 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:56:29.255477 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.255396 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-feast-data\") pod \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " Jul 30 06:56:29.255477 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.255453 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-ui-tls\") pod \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " Jul 30 06:56:29.255477 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.255471 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-online-tls\") pod \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " Jul 30 06:56:29.255909 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.255533 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-offline-tls\") pod \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " Jul 30 06:56:29.255909 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.255555 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-registry-tls\") pod \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " Jul 30 06:56:29.255909 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.255585 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9x7nz\" (UniqueName: \"kubernetes.io/projected/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-kube-api-access-9x7nz\") pod \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\" (UID: \"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7\") " Jul 30 06:56:29.256053 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.255986 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-feast-data" (OuterVolumeSpecName: "feast-data") pod "ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" (UID: "ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 06:56:29.258161 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.258108 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" (UID: "ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:56:29.258161 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.258151 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-online-tls" (OuterVolumeSpecName: "online-tls") pod "ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" (UID: "ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:56:29.258344 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.258184 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" (UID: "ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:56:29.258400 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.258379 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" (UID: "ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:56:29.258476 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.258458 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-kube-api-access-9x7nz" (OuterVolumeSpecName: "kube-api-access-9x7nz") pod "ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" (UID: "ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7"). InnerVolumeSpecName "kube-api-access-9x7nz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 06:56:29.356353 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.356320 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-feast-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:56:29.356353 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.356353 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-ui-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:56:29.356605 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.356367 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-online-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:56:29.356605 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.356380 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-offline-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:56:29.356605 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.356392 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-registry-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:56:29.356605 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:29.356404 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9x7nz\" (UniqueName: \"kubernetes.io/projected/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7-kube-api-access-9x7nz\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:56:30.015328 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:30.015303 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" Jul 30 06:56:30.015328 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:30.015305 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr" event={"ID":"ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7","Type":"ContainerDied","Data":"d977b8a5b2d5a5989d42914b5888fe4e6335633c1c1f063dce9a2ae987a1e7ee"} Jul 30 06:56:30.015826 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:30.015383 2575 scope.go:117] "RemoveContainer" containerID="d2c5ed9c186d63d9b32c127665f4893b06b621e61cf8fe6b1fd13b05e96b59c7" Jul 30 06:56:30.024570 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:30.024549 2575 scope.go:117] "RemoveContainer" containerID="6f8eefc72825f8bd73663d3f60ddffffb024a26204342f68c2d065961707a544" Jul 30 06:56:30.033806 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:30.033781 2575 scope.go:117] "RemoveContainer" containerID="0b0e902bfb80c07fb51a2d3353e39e77aeb891414e6f404798c9f36273242488" Jul 30 06:56:30.039454 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:30.039428 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr"] Jul 30 06:56:30.043096 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:30.043074 2575 scope.go:117] "RemoveContainer" containerID="d6423ec9a4d241ca997dac940246781ff51f5686412ccf46b7f6e4aa6e71f81c" Jul 30 06:56:30.044177 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:30.044153 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b7885d7d4-kfkdr"] Jul 30 06:56:30.051395 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:30.051369 2575 scope.go:117] "RemoveContainer" containerID="fa0878cf65ff0c1e54f0ab1c2712ea86259bbf87fe1575cebb0688fd7d22d5d5" Jul 30 06:56:30.058470 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:30.058453 2575 scope.go:117] "RemoveContainer" containerID="f3f4de490cd934747e13af87d1e8d384822f1830b917f97d85c4d01ac5f10627" Jul 30 06:56:31.175959 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:56:31.175930 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" path="/var/lib/kubelet/pods/ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7/volumes" Jul 30 06:57:22.514570 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.514540 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7"] Jul 30 06:57:22.515001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.514859 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="online" Jul 30 06:57:22.515001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.514869 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="online" Jul 30 06:57:22.515001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.514880 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="registry" Jul 30 06:57:22.515001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.514885 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="registry" Jul 30 06:57:22.515001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.514896 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="ui" Jul 30 06:57:22.515001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.514902 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="ui" Jul 30 06:57:22.515001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.514915 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="offline" Jul 30 06:57:22.515001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.514920 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="offline" Jul 30 06:57:22.515001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.514927 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31175bf8-8013-4336-9be6-0bc787b46884" containerName="console" Jul 30 06:57:22.515001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.514932 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="31175bf8-8013-4336-9be6-0bc787b46884" containerName="console" Jul 30 06:57:22.515001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.514940 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="feast-init" Jul 30 06:57:22.515001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.514945 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="feast-init" Jul 30 06:57:22.515001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.514952 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="feast-apply" Jul 30 06:57:22.515001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.514958 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="feast-apply" Jul 30 06:57:22.515001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.515001 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="online" Jul 30 06:57:22.515447 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.515011 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="31175bf8-8013-4336-9be6-0bc787b46884" containerName="console" Jul 30 06:57:22.515447 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.515018 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="offline" Jul 30 06:57:22.515447 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.515026 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="ui" Jul 30 06:57:22.515447 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.515031 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab9fc9d7-fbae-472a-99ef-0a7c27fa65a7" containerName="registry" Jul 30 06:57:22.519716 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.519695 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.524349 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.524318 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jul 30 06:57:22.524480 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.524360 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Jul 30 06:57:22.524480 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.524384 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jul 30 06:57:22.524480 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.524330 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jul 30 06:57:22.524673 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.524617 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jul 30 06:57:22.524743 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.524675 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jul 30 06:57:22.524743 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.524695 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jul 30 06:57:22.525333 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.525316 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-d8rnl\"" Jul 30 06:57:22.538255 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.538230 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7"] Jul 30 06:57:22.595337 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.595305 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-online-tls\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.595533 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.595344 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d8a2be61-1817-4041-b423-61512b737c36-feast-data\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.595533 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.595390 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-ui-tls\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.595533 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.595414 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-offline-tls\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.595533 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.595496 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zswq6\" (UniqueName: \"kubernetes.io/projected/d8a2be61-1817-4041-b423-61512b737c36-kube-api-access-zswq6\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.595764 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.595547 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d8a2be61-1817-4041-b423-61512b737c36-registry-tls\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.696594 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.696559 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-offline-tls\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.696783 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.696610 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zswq6\" (UniqueName: \"kubernetes.io/projected/d8a2be61-1817-4041-b423-61512b737c36-kube-api-access-zswq6\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.696783 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.696639 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d8a2be61-1817-4041-b423-61512b737c36-registry-tls\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.696783 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.696696 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-online-tls\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.696783 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.696714 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d8a2be61-1817-4041-b423-61512b737c36-feast-data\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.696783 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.696755 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-ui-tls\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.697194 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.697161 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d8a2be61-1817-4041-b423-61512b737c36-feast-data\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.697450 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.697425 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d8a2be61-1817-4041-b423-61512b737c36-registry-tls\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.699241 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.699220 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-online-tls\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.699337 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.699310 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-offline-tls\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.699393 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.699345 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-ui-tls\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.705685 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.705639 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zswq6\" (UniqueName: \"kubernetes.io/projected/d8a2be61-1817-4041-b423-61512b737c36-kube-api-access-zswq6\") pod \"feast-simple-feast-remote-setup-5574dc4b75-9qmr7\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.829130 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.829055 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:22.973166 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:22.973134 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7"] Jul 30 06:57:22.976816 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:57:22.976790 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8a2be61_1817_4041_b423_61512b737c36.slice/crio-1eb07c4aafefc8b600ae5d3bf1a1d8f2547278fccb058a88b396f8d429c8fcfc WatchSource:0}: Error finding container 1eb07c4aafefc8b600ae5d3bf1a1d8f2547278fccb058a88b396f8d429c8fcfc: Status 404 returned error can't find the container with id 1eb07c4aafefc8b600ae5d3bf1a1d8f2547278fccb058a88b396f8d429c8fcfc Jul 30 06:57:23.227695 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:23.227625 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" event={"ID":"d8a2be61-1817-4041-b423-61512b737c36","Type":"ContainerStarted","Data":"2a4f492df96ecefdae6133e95338ae1c8baa48d9514dd3f01b98cf0ed4b626ef"} Jul 30 06:57:23.227896 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:23.227701 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" event={"ID":"d8a2be61-1817-4041-b423-61512b737c36","Type":"ContainerStarted","Data":"1eb07c4aafefc8b600ae5d3bf1a1d8f2547278fccb058a88b396f8d429c8fcfc"} Jul 30 06:57:29.259160 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:29.259128 2575 generic.go:356] "Generic (PLEG): container finished" podID="d8a2be61-1817-4041-b423-61512b737c36" containerID="2a4f492df96ecefdae6133e95338ae1c8baa48d9514dd3f01b98cf0ed4b626ef" exitCode=0 Jul 30 06:57:29.259734 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:29.259208 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" event={"ID":"d8a2be61-1817-4041-b423-61512b737c36","Type":"ContainerDied","Data":"2a4f492df96ecefdae6133e95338ae1c8baa48d9514dd3f01b98cf0ed4b626ef"} Jul 30 06:57:30.265382 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:30.265339 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" event={"ID":"d8a2be61-1817-4041-b423-61512b737c36","Type":"ContainerStarted","Data":"24947382201ef2f98023bb07430cf697e4908e53aa1ec7fa8a7bbb6df6942a30"} Jul 30 06:57:35.286560 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:35.286527 2575 generic.go:356] "Generic (PLEG): container finished" podID="d8a2be61-1817-4041-b423-61512b737c36" containerID="24947382201ef2f98023bb07430cf697e4908e53aa1ec7fa8a7bbb6df6942a30" exitCode=0 Jul 30 06:57:35.286952 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:35.286593 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" event={"ID":"d8a2be61-1817-4041-b423-61512b737c36","Type":"ContainerDied","Data":"24947382201ef2f98023bb07430cf697e4908e53aa1ec7fa8a7bbb6df6942a30"} Jul 30 06:57:36.296879 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:36.296832 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" event={"ID":"d8a2be61-1817-4041-b423-61512b737c36","Type":"ContainerStarted","Data":"a26d2f7ebabca293c712b38128f99f8f1a5d213125978ac95289b4d9be56b357"} Jul 30 06:57:36.297378 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:36.296899 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" event={"ID":"d8a2be61-1817-4041-b423-61512b737c36","Type":"ContainerStarted","Data":"7b20c61a7b3005635bac0b917879b22134a92c97823f51ae0f13590edf244b9a"} Jul 30 06:57:36.297378 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:36.296921 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" event={"ID":"d8a2be61-1817-4041-b423-61512b737c36","Type":"ContainerStarted","Data":"668eec297508bff198ba08b33f1100f1549c4abc5070bb48789db8e9c16fde22"} Jul 30 06:57:37.829593 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:37.829554 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:37.830099 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:37.829706 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:37.830099 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:37.829732 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:37.831472 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:37.831442 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Jul 30 06:57:37.831621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:37.831439 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.28:8816: connect: connection refused" Jul 30 06:57:37.831722 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:37.831492 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Jul 30 06:57:40.830197 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:40.830143 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Jul 30 06:57:40.830197 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:40.830172 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Jul 30 06:57:40.830638 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:40.830158 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.28:8816: connect: connection refused" Jul 30 06:57:42.829497 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:42.829464 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:42.829943 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:42.829513 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:42.829943 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:42.829525 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:43.830787 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:43.830755 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:43.831203 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:43.830888 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:43.831525 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:43.831505 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:43.831797 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:43.831778 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:43.837603 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:43.837578 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:57:43.873818 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:43.873761 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" podStartSLOduration=21.873742311 podStartE2EDuration="21.873742311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:57:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:57:36.334049359 +0000 UTC m=+817.691053524" watchObservedRunningTime="2026-07-30 06:57:43.873742311 +0000 UTC m=+825.230746473" Jul 30 06:57:44.337549 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:57:44.337512 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:58:45.767419 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:45.767385 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7"] Jul 30 06:58:45.767965 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:45.767676 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="online" containerID="cri-o://668eec297508bff198ba08b33f1100f1549c4abc5070bb48789db8e9c16fde22" gracePeriod=30 Jul 30 06:58:45.767965 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:45.767753 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="ui" containerID="cri-o://a26d2f7ebabca293c712b38128f99f8f1a5d213125978ac95289b4d9be56b357" gracePeriod=30 Jul 30 06:58:45.767965 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:45.767753 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="offline" containerID="cri-o://7b20c61a7b3005635bac0b917879b22134a92c97823f51ae0f13590edf244b9a" gracePeriod=30 Jul 30 06:58:45.884269 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:45.884230 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs"] Jul 30 06:58:45.884513 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:45.884481 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="registry" containerID="cri-o://605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c" gracePeriod=30 Jul 30 06:58:45.884646 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:45.884580 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="online" containerID="cri-o://b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1" gracePeriod=30 Jul 30 06:58:45.884646 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:45.884593 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="ui" containerID="cri-o://78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462" gracePeriod=30 Jul 30 06:58:45.884820 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:45.884571 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="offline" containerID="cri-o://4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3" gracePeriod=30 Jul 30 06:58:46.587908 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:46.587878 2575 generic.go:356] "Generic (PLEG): container finished" podID="142df858-460c-4651-ba54-577c0f134c2a" containerID="78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462" exitCode=0 Jul 30 06:58:46.588052 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:46.587957 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" event={"ID":"142df858-460c-4651-ba54-577c0f134c2a","Type":"ContainerDied","Data":"78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462"} Jul 30 06:58:46.590897 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:46.590875 2575 generic.go:356] "Generic (PLEG): container finished" podID="d8a2be61-1817-4041-b423-61512b737c36" containerID="a26d2f7ebabca293c712b38128f99f8f1a5d213125978ac95289b4d9be56b357" exitCode=0 Jul 30 06:58:46.590897 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:46.590894 2575 generic.go:356] "Generic (PLEG): container finished" podID="d8a2be61-1817-4041-b423-61512b737c36" containerID="668eec297508bff198ba08b33f1100f1549c4abc5070bb48789db8e9c16fde22" exitCode=0 Jul 30 06:58:46.591040 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:46.590944 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" event={"ID":"d8a2be61-1817-4041-b423-61512b737c36","Type":"ContainerDied","Data":"a26d2f7ebabca293c712b38128f99f8f1a5d213125978ac95289b4d9be56b357"} Jul 30 06:58:46.591040 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:46.590983 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" event={"ID":"d8a2be61-1817-4041-b423-61512b737c36","Type":"ContainerDied","Data":"668eec297508bff198ba08b33f1100f1549c4abc5070bb48789db8e9c16fde22"} Jul 30 06:58:47.598114 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:47.598032 2575 generic.go:356] "Generic (PLEG): container finished" podID="142df858-460c-4651-ba54-577c0f134c2a" containerID="b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1" exitCode=0 Jul 30 06:58:47.598443 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:47.598106 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" event={"ID":"142df858-460c-4651-ba54-577c0f134c2a","Type":"ContainerDied","Data":"b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1"} Jul 30 06:58:50.602405 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:50.602303 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Jul 30 06:58:50.970433 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:50.970393 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jul 30 06:58:53.832249 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:53.832206 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Jul 30 06:58:54.333521 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:54.333480 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Jul 30 06:58:59.265941 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:59.265829 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/1.log" Jul 30 06:58:59.269917 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:59.266832 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/1.log" Jul 30 06:58:59.272271 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:59.272250 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 06:58:59.273131 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:58:59.273113 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 06:59:00.602886 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:00.602843 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Jul 30 06:59:00.970415 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:00.970374 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jul 30 06:59:03.831707 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:03.831641 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Jul 30 06:59:04.333593 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:04.333543 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Jul 30 06:59:10.602408 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:10.602358 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Jul 30 06:59:10.602865 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:10.602527 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:59:10.970806 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:10.970766 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jul 30 06:59:10.971011 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:10.970875 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:59:13.831501 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:13.831456 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Jul 30 06:59:13.832030 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:13.831601 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:59:14.334001 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:14.333963 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Jul 30 06:59:14.334196 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:14.334070 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:59:15.985999 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:15.985975 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:59:16.057127 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.057034 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d8a2be61-1817-4041-b423-61512b737c36-registry-tls\") pod \"d8a2be61-1817-4041-b423-61512b737c36\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " Jul 30 06:59:16.057127 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.057091 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-ui-tls\") pod \"d8a2be61-1817-4041-b423-61512b737c36\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " Jul 30 06:59:16.057358 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.057197 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d8a2be61-1817-4041-b423-61512b737c36-feast-data\") pod \"d8a2be61-1817-4041-b423-61512b737c36\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " Jul 30 06:59:16.057358 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.057245 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-online-tls\") pod \"d8a2be61-1817-4041-b423-61512b737c36\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " Jul 30 06:59:16.057358 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.057281 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-offline-tls\") pod \"d8a2be61-1817-4041-b423-61512b737c36\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " Jul 30 06:59:16.057358 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.057326 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zswq6\" (UniqueName: \"kubernetes.io/projected/d8a2be61-1817-4041-b423-61512b737c36-kube-api-access-zswq6\") pod \"d8a2be61-1817-4041-b423-61512b737c36\" (UID: \"d8a2be61-1817-4041-b423-61512b737c36\") " Jul 30 06:59:16.057565 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.057420 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8a2be61-1817-4041-b423-61512b737c36-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d8a2be61-1817-4041-b423-61512b737c36" (UID: "d8a2be61-1817-4041-b423-61512b737c36"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 06:59:16.057621 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.057598 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d8a2be61-1817-4041-b423-61512b737c36-registry-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:59:16.057702 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.057684 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8a2be61-1817-4041-b423-61512b737c36-feast-data" (OuterVolumeSpecName: "feast-data") pod "d8a2be61-1817-4041-b423-61512b737c36" (UID: "d8a2be61-1817-4041-b423-61512b737c36"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 06:59:16.059538 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.059504 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "d8a2be61-1817-4041-b423-61512b737c36" (UID: "d8a2be61-1817-4041-b423-61512b737c36"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:59:16.059704 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.059554 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-online-tls" (OuterVolumeSpecName: "online-tls") pod "d8a2be61-1817-4041-b423-61512b737c36" (UID: "d8a2be61-1817-4041-b423-61512b737c36"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:59:16.059704 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.059565 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "d8a2be61-1817-4041-b423-61512b737c36" (UID: "d8a2be61-1817-4041-b423-61512b737c36"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:59:16.059704 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.059642 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8a2be61-1817-4041-b423-61512b737c36-kube-api-access-zswq6" (OuterVolumeSpecName: "kube-api-access-zswq6") pod "d8a2be61-1817-4041-b423-61512b737c36" (UID: "d8a2be61-1817-4041-b423-61512b737c36"). InnerVolumeSpecName "kube-api-access-zswq6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 06:59:16.158742 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.158700 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d8a2be61-1817-4041-b423-61512b737c36-feast-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:59:16.158935 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.158753 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-online-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:59:16.158935 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.158763 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-offline-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:59:16.158935 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.158772 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zswq6\" (UniqueName: \"kubernetes.io/projected/d8a2be61-1817-4041-b423-61512b737c36-kube-api-access-zswq6\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:59:16.158935 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.158781 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d8a2be61-1817-4041-b423-61512b737c36-ui-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:59:16.528103 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.528080 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:59:16.562211 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.562185 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-online-tls\") pod \"142df858-460c-4651-ba54-577c0f134c2a\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " Jul 30 06:59:16.562380 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.562279 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-registry-tls\") pod \"142df858-460c-4651-ba54-577c0f134c2a\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " Jul 30 06:59:16.562380 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.562307 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-ui-tls\") pod \"142df858-460c-4651-ba54-577c0f134c2a\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " Jul 30 06:59:16.562380 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.562329 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-offline-tls\") pod \"142df858-460c-4651-ba54-577c0f134c2a\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " Jul 30 06:59:16.562380 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.562369 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/142df858-460c-4651-ba54-577c0f134c2a-feast-data\") pod \"142df858-460c-4651-ba54-577c0f134c2a\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " Jul 30 06:59:16.562599 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.562407 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbqrw\" (UniqueName: \"kubernetes.io/projected/142df858-460c-4651-ba54-577c0f134c2a-kube-api-access-jbqrw\") pod \"142df858-460c-4651-ba54-577c0f134c2a\" (UID: \"142df858-460c-4651-ba54-577c0f134c2a\") " Jul 30 06:59:16.563372 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.563001 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/142df858-460c-4651-ba54-577c0f134c2a-feast-data" (OuterVolumeSpecName: "feast-data") pod "142df858-460c-4651-ba54-577c0f134c2a" (UID: "142df858-460c-4651-ba54-577c0f134c2a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 06:59:16.563615 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.563525 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/142df858-460c-4651-ba54-577c0f134c2a-feast-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:59:16.565032 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.565005 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "142df858-460c-4651-ba54-577c0f134c2a" (UID: "142df858-460c-4651-ba54-577c0f134c2a"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:59:16.565122 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.565062 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-online-tls" (OuterVolumeSpecName: "online-tls") pod "142df858-460c-4651-ba54-577c0f134c2a" (UID: "142df858-460c-4651-ba54-577c0f134c2a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:59:16.565167 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.565149 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "142df858-460c-4651-ba54-577c0f134c2a" (UID: "142df858-460c-4651-ba54-577c0f134c2a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:59:16.565205 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.565165 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "142df858-460c-4651-ba54-577c0f134c2a" (UID: "142df858-460c-4651-ba54-577c0f134c2a"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 06:59:16.565345 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.565319 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/142df858-460c-4651-ba54-577c0f134c2a-kube-api-access-jbqrw" (OuterVolumeSpecName: "kube-api-access-jbqrw") pod "142df858-460c-4651-ba54-577c0f134c2a" (UID: "142df858-460c-4651-ba54-577c0f134c2a"). InnerVolumeSpecName "kube-api-access-jbqrw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 06:59:16.664454 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.664361 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-registry-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:59:16.664454 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.664398 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-ui-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:59:16.664454 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.664406 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-offline-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:59:16.664454 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.664416 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbqrw\" (UniqueName: \"kubernetes.io/projected/142df858-460c-4651-ba54-577c0f134c2a-kube-api-access-jbqrw\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:59:16.664454 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.664425 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/142df858-460c-4651-ba54-577c0f134c2a-online-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 06:59:16.724564 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.724530 2575 generic.go:356] "Generic (PLEG): container finished" podID="d8a2be61-1817-4041-b423-61512b737c36" containerID="7b20c61a7b3005635bac0b917879b22134a92c97823f51ae0f13590edf244b9a" exitCode=137 Jul 30 06:59:16.724735 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.724592 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" Jul 30 06:59:16.724735 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.724621 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" event={"ID":"d8a2be61-1817-4041-b423-61512b737c36","Type":"ContainerDied","Data":"7b20c61a7b3005635bac0b917879b22134a92c97823f51ae0f13590edf244b9a"} Jul 30 06:59:16.724735 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.724687 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7" event={"ID":"d8a2be61-1817-4041-b423-61512b737c36","Type":"ContainerDied","Data":"1eb07c4aafefc8b600ae5d3bf1a1d8f2547278fccb058a88b396f8d429c8fcfc"} Jul 30 06:59:16.724735 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.724714 2575 scope.go:117] "RemoveContainer" containerID="a26d2f7ebabca293c712b38128f99f8f1a5d213125978ac95289b4d9be56b357" Jul 30 06:59:16.727742 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.727717 2575 generic.go:356] "Generic (PLEG): container finished" podID="142df858-460c-4651-ba54-577c0f134c2a" containerID="4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3" exitCode=137 Jul 30 06:59:16.727742 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.727742 2575 generic.go:356] "Generic (PLEG): container finished" podID="142df858-460c-4651-ba54-577c0f134c2a" containerID="605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c" exitCode=137 Jul 30 06:59:16.727895 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.727771 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" event={"ID":"142df858-460c-4651-ba54-577c0f134c2a","Type":"ContainerDied","Data":"4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3"} Jul 30 06:59:16.727895 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.727797 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" Jul 30 06:59:16.727895 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.727808 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" event={"ID":"142df858-460c-4651-ba54-577c0f134c2a","Type":"ContainerDied","Data":"605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c"} Jul 30 06:59:16.727895 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.727829 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs" event={"ID":"142df858-460c-4651-ba54-577c0f134c2a","Type":"ContainerDied","Data":"87dae8d4219665c3b46f59979fe27cacd1773ed6d9351c1d29db7d85fb7ec052"} Jul 30 06:59:16.735017 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.734999 2575 scope.go:117] "RemoveContainer" containerID="7b20c61a7b3005635bac0b917879b22134a92c97823f51ae0f13590edf244b9a" Jul 30 06:59:16.742716 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.742697 2575 scope.go:117] "RemoveContainer" containerID="668eec297508bff198ba08b33f1100f1549c4abc5070bb48789db8e9c16fde22" Jul 30 06:59:16.749608 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.749587 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7"] Jul 30 06:59:16.751027 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.751006 2575 scope.go:117] "RemoveContainer" containerID="24947382201ef2f98023bb07430cf697e4908e53aa1ec7fa8a7bbb6df6942a30" Jul 30 06:59:16.754090 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.754069 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5574dc4b75-9qmr7"] Jul 30 06:59:16.758689 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.758648 2575 scope.go:117] "RemoveContainer" containerID="2a4f492df96ecefdae6133e95338ae1c8baa48d9514dd3f01b98cf0ed4b626ef" Jul 30 06:59:16.765617 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.765596 2575 scope.go:117] "RemoveContainer" containerID="a26d2f7ebabca293c712b38128f99f8f1a5d213125978ac95289b4d9be56b357" Jul 30 06:59:16.765960 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:59:16.765937 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a26d2f7ebabca293c712b38128f99f8f1a5d213125978ac95289b4d9be56b357\": container with ID starting with a26d2f7ebabca293c712b38128f99f8f1a5d213125978ac95289b4d9be56b357 not found: ID does not exist" containerID="a26d2f7ebabca293c712b38128f99f8f1a5d213125978ac95289b4d9be56b357" Jul 30 06:59:16.766215 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.765973 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a26d2f7ebabca293c712b38128f99f8f1a5d213125978ac95289b4d9be56b357"} err="failed to get container status \"a26d2f7ebabca293c712b38128f99f8f1a5d213125978ac95289b4d9be56b357\": rpc error: code = NotFound desc = could not find container \"a26d2f7ebabca293c712b38128f99f8f1a5d213125978ac95289b4d9be56b357\": container with ID starting with a26d2f7ebabca293c712b38128f99f8f1a5d213125978ac95289b4d9be56b357 not found: ID does not exist" Jul 30 06:59:16.766215 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.765998 2575 scope.go:117] "RemoveContainer" containerID="7b20c61a7b3005635bac0b917879b22134a92c97823f51ae0f13590edf244b9a" Jul 30 06:59:16.766801 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:59:16.766313 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b20c61a7b3005635bac0b917879b22134a92c97823f51ae0f13590edf244b9a\": container with ID starting with 7b20c61a7b3005635bac0b917879b22134a92c97823f51ae0f13590edf244b9a not found: ID does not exist" containerID="7b20c61a7b3005635bac0b917879b22134a92c97823f51ae0f13590edf244b9a" Jul 30 06:59:16.766801 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.766341 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b20c61a7b3005635bac0b917879b22134a92c97823f51ae0f13590edf244b9a"} err="failed to get container status \"7b20c61a7b3005635bac0b917879b22134a92c97823f51ae0f13590edf244b9a\": rpc error: code = NotFound desc = could not find container \"7b20c61a7b3005635bac0b917879b22134a92c97823f51ae0f13590edf244b9a\": container with ID starting with 7b20c61a7b3005635bac0b917879b22134a92c97823f51ae0f13590edf244b9a not found: ID does not exist" Jul 30 06:59:16.766801 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.766362 2575 scope.go:117] "RemoveContainer" containerID="668eec297508bff198ba08b33f1100f1549c4abc5070bb48789db8e9c16fde22" Jul 30 06:59:16.766946 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:59:16.766822 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"668eec297508bff198ba08b33f1100f1549c4abc5070bb48789db8e9c16fde22\": container with ID starting with 668eec297508bff198ba08b33f1100f1549c4abc5070bb48789db8e9c16fde22 not found: ID does not exist" containerID="668eec297508bff198ba08b33f1100f1549c4abc5070bb48789db8e9c16fde22" Jul 30 06:59:16.766946 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.766847 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"668eec297508bff198ba08b33f1100f1549c4abc5070bb48789db8e9c16fde22"} err="failed to get container status \"668eec297508bff198ba08b33f1100f1549c4abc5070bb48789db8e9c16fde22\": rpc error: code = NotFound desc = could not find container \"668eec297508bff198ba08b33f1100f1549c4abc5070bb48789db8e9c16fde22\": container with ID starting with 668eec297508bff198ba08b33f1100f1549c4abc5070bb48789db8e9c16fde22 not found: ID does not exist" Jul 30 06:59:16.766946 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.766862 2575 scope.go:117] "RemoveContainer" containerID="24947382201ef2f98023bb07430cf697e4908e53aa1ec7fa8a7bbb6df6942a30" Jul 30 06:59:16.767140 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:59:16.767117 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24947382201ef2f98023bb07430cf697e4908e53aa1ec7fa8a7bbb6df6942a30\": container with ID starting with 24947382201ef2f98023bb07430cf697e4908e53aa1ec7fa8a7bbb6df6942a30 not found: ID does not exist" containerID="24947382201ef2f98023bb07430cf697e4908e53aa1ec7fa8a7bbb6df6942a30" Jul 30 06:59:16.767182 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.767149 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24947382201ef2f98023bb07430cf697e4908e53aa1ec7fa8a7bbb6df6942a30"} err="failed to get container status \"24947382201ef2f98023bb07430cf697e4908e53aa1ec7fa8a7bbb6df6942a30\": rpc error: code = NotFound desc = could not find container \"24947382201ef2f98023bb07430cf697e4908e53aa1ec7fa8a7bbb6df6942a30\": container with ID starting with 24947382201ef2f98023bb07430cf697e4908e53aa1ec7fa8a7bbb6df6942a30 not found: ID does not exist" Jul 30 06:59:16.767182 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.767171 2575 scope.go:117] "RemoveContainer" containerID="2a4f492df96ecefdae6133e95338ae1c8baa48d9514dd3f01b98cf0ed4b626ef" Jul 30 06:59:16.767443 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:59:16.767412 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a4f492df96ecefdae6133e95338ae1c8baa48d9514dd3f01b98cf0ed4b626ef\": container with ID starting with 2a4f492df96ecefdae6133e95338ae1c8baa48d9514dd3f01b98cf0ed4b626ef not found: ID does not exist" containerID="2a4f492df96ecefdae6133e95338ae1c8baa48d9514dd3f01b98cf0ed4b626ef" Jul 30 06:59:16.767523 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.767452 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a4f492df96ecefdae6133e95338ae1c8baa48d9514dd3f01b98cf0ed4b626ef"} err="failed to get container status \"2a4f492df96ecefdae6133e95338ae1c8baa48d9514dd3f01b98cf0ed4b626ef\": rpc error: code = NotFound desc = could not find container \"2a4f492df96ecefdae6133e95338ae1c8baa48d9514dd3f01b98cf0ed4b626ef\": container with ID starting with 2a4f492df96ecefdae6133e95338ae1c8baa48d9514dd3f01b98cf0ed4b626ef not found: ID does not exist" Jul 30 06:59:16.767523 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.767476 2575 scope.go:117] "RemoveContainer" containerID="78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462" Jul 30 06:59:16.768286 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.768266 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs"] Jul 30 06:59:16.775476 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.775454 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-7b7885d7d4-x5sbs"] Jul 30 06:59:16.775977 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.775880 2575 scope.go:117] "RemoveContainer" containerID="4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3" Jul 30 06:59:16.783834 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.783818 2575 scope.go:117] "RemoveContainer" containerID="b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1" Jul 30 06:59:16.798540 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.798522 2575 scope.go:117] "RemoveContainer" containerID="605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c" Jul 30 06:59:16.805959 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.805940 2575 scope.go:117] "RemoveContainer" containerID="ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490" Jul 30 06:59:16.815981 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.815963 2575 scope.go:117] "RemoveContainer" containerID="41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe" Jul 30 06:59:16.823239 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.823221 2575 scope.go:117] "RemoveContainer" containerID="78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462" Jul 30 06:59:16.823504 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:59:16.823489 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462\": container with ID starting with 78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462 not found: ID does not exist" containerID="78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462" Jul 30 06:59:16.823549 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.823512 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462"} err="failed to get container status \"78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462\": rpc error: code = NotFound desc = could not find container \"78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462\": container with ID starting with 78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462 not found: ID does not exist" Jul 30 06:59:16.823549 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.823530 2575 scope.go:117] "RemoveContainer" containerID="4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3" Jul 30 06:59:16.823756 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:59:16.823741 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3\": container with ID starting with 4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3 not found: ID does not exist" containerID="4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3" Jul 30 06:59:16.823808 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.823757 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3"} err="failed to get container status \"4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3\": rpc error: code = NotFound desc = could not find container \"4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3\": container with ID starting with 4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3 not found: ID does not exist" Jul 30 06:59:16.823808 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.823768 2575 scope.go:117] "RemoveContainer" containerID="b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1" Jul 30 06:59:16.823971 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:59:16.823953 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1\": container with ID starting with b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1 not found: ID does not exist" containerID="b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1" Jul 30 06:59:16.824010 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.823978 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1"} err="failed to get container status \"b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1\": rpc error: code = NotFound desc = could not find container \"b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1\": container with ID starting with b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1 not found: ID does not exist" Jul 30 06:59:16.824010 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.823996 2575 scope.go:117] "RemoveContainer" containerID="605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c" Jul 30 06:59:16.824203 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:59:16.824187 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c\": container with ID starting with 605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c not found: ID does not exist" containerID="605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c" Jul 30 06:59:16.824243 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.824208 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c"} err="failed to get container status \"605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c\": rpc error: code = NotFound desc = could not find container \"605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c\": container with ID starting with 605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c not found: ID does not exist" Jul 30 06:59:16.824243 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.824219 2575 scope.go:117] "RemoveContainer" containerID="ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490" Jul 30 06:59:16.824420 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:59:16.824406 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490\": container with ID starting with ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490 not found: ID does not exist" containerID="ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490" Jul 30 06:59:16.824469 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.824423 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490"} err="failed to get container status \"ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490\": rpc error: code = NotFound desc = could not find container \"ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490\": container with ID starting with ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490 not found: ID does not exist" Jul 30 06:59:16.824469 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.824436 2575 scope.go:117] "RemoveContainer" containerID="41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe" Jul 30 06:59:16.824635 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:59:16.824621 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe\": container with ID starting with 41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe not found: ID does not exist" containerID="41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe" Jul 30 06:59:16.824705 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.824639 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe"} err="failed to get container status \"41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe\": rpc error: code = NotFound desc = could not find container \"41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe\": container with ID starting with 41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe not found: ID does not exist" Jul 30 06:59:16.824705 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.824668 2575 scope.go:117] "RemoveContainer" containerID="78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462" Jul 30 06:59:16.824919 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.824890 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462"} err="failed to get container status \"78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462\": rpc error: code = NotFound desc = could not find container \"78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462\": container with ID starting with 78f59215bb74b4a6b21dbe46af1cb90ee144f0ccefbc26ef39c22a8d1c72d462 not found: ID does not exist" Jul 30 06:59:16.824919 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.824910 2575 scope.go:117] "RemoveContainer" containerID="4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3" Jul 30 06:59:16.825111 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.825095 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3"} err="failed to get container status \"4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3\": rpc error: code = NotFound desc = could not find container \"4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3\": container with ID starting with 4bd72fd9e87521c5b3c05bb1edde7e0f2c2f0fd1ce3af490ceb3964d325eafd3 not found: ID does not exist" Jul 30 06:59:16.825163 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.825112 2575 scope.go:117] "RemoveContainer" containerID="b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1" Jul 30 06:59:16.825337 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.825322 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1"} err="failed to get container status \"b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1\": rpc error: code = NotFound desc = could not find container \"b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1\": container with ID starting with b4427263e54e95ccadb3e073e9923c561fa3f0af9418a73f340e3ab6f59cabb1 not found: ID does not exist" Jul 30 06:59:16.825378 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.825337 2575 scope.go:117] "RemoveContainer" containerID="605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c" Jul 30 06:59:16.825525 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.825507 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c"} err="failed to get container status \"605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c\": rpc error: code = NotFound desc = could not find container \"605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c\": container with ID starting with 605c4ef45b127aea29b875d00ae7cad5ad294e9d97c48e42ba3c6e397611422c not found: ID does not exist" Jul 30 06:59:16.825595 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.825528 2575 scope.go:117] "RemoveContainer" containerID="ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490" Jul 30 06:59:16.825763 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.825742 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490"} err="failed to get container status \"ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490\": rpc error: code = NotFound desc = could not find container \"ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490\": container with ID starting with ca657abfa12d47321db71ef5409b343e08b2703619be6b158086869325130490 not found: ID does not exist" Jul 30 06:59:16.825813 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.825760 2575 scope.go:117] "RemoveContainer" containerID="41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe" Jul 30 06:59:16.825954 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:16.825939 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe"} err="failed to get container status \"41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe\": rpc error: code = NotFound desc = could not find container \"41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe\": container with ID starting with 41961d18334c7bc8c1db5bcdc6e384d966faf6fb01958c99f325c703c2a2b7fe not found: ID does not exist" Jul 30 06:59:17.175998 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:17.175960 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="142df858-460c-4651-ba54-577c0f134c2a" path="/var/lib/kubelet/pods/142df858-460c-4651-ba54-577c0f134c2a/volumes" Jul 30 06:59:17.176755 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:17.176735 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8a2be61-1817-4041-b423-61512b737c36" path="/var/lib/kubelet/pods/d8a2be61-1817-4041-b423-61512b737c36/volumes" Jul 30 06:59:29.321674 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.321630 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-qcpsz"] Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.321944 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="ui" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.321956 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="ui" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.321970 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="online" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.321976 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="online" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.321985 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="feast-apply" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.321990 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="feast-apply" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.321999 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="feast-init" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322004 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="feast-init" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322010 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="feast-apply" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322015 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="feast-apply" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322023 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="feast-init" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322031 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="feast-init" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322038 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="offline" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322043 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="offline" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322049 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="offline" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322054 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="offline" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322063 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="registry" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322067 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="registry" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322073 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="online" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322078 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="online" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322084 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="ui" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322089 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="ui" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322134 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="online" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322142 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="ui" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322149 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="offline" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322154 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="registry" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322158 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8a2be61-1817-4041-b423-61512b737c36" containerName="offline" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322185 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="ui" Jul 30 06:59:29.322247 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.322192 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="142df858-460c-4651-ba54-577c0f134c2a" containerName="online" Jul 30 06:59:29.324905 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.324886 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-qcpsz" Jul 30 06:59:29.332006 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.331957 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 30 06:59:29.332351 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.332334 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-8c6bw\"" Jul 30 06:59:29.333711 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.333695 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Jul 30 06:59:29.337349 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.337332 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 30 06:59:29.344610 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.344590 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-qcpsz"] Jul 30 06:59:29.423034 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.423003 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-dczqb"] Jul 30 06:59:29.426084 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.426065 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-dczqb" Jul 30 06:59:29.463402 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.463364 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-dczqb"] Jul 30 06:59:29.464815 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.464791 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6v88\" (UniqueName: \"kubernetes.io/projected/bac18c68-88dc-4e93-8f7e-61df12c19072-kube-api-access-v6v88\") pod \"postgres-864846d4f5-qcpsz\" (UID: \"bac18c68-88dc-4e93-8f7e-61df12c19072\") " pod="test-ns-feast/postgres-864846d4f5-qcpsz" Jul 30 06:59:29.464931 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.464911 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/bac18c68-88dc-4e93-8f7e-61df12c19072-postgresdata\") pod \"postgres-864846d4f5-qcpsz\" (UID: \"bac18c68-88dc-4e93-8f7e-61df12c19072\") " pod="test-ns-feast/postgres-864846d4f5-qcpsz" Jul 30 06:59:29.565631 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.565591 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59srp\" (UniqueName: \"kubernetes.io/projected/899c8706-8461-484e-b25d-b41b26f5c38d-kube-api-access-59srp\") pod \"redis-69bbd798b6-dczqb\" (UID: \"899c8706-8461-484e-b25d-b41b26f5c38d\") " pod="test-ns-feast/redis-69bbd798b6-dczqb" Jul 30 06:59:29.565866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.565677 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/bac18c68-88dc-4e93-8f7e-61df12c19072-postgresdata\") pod \"postgres-864846d4f5-qcpsz\" (UID: \"bac18c68-88dc-4e93-8f7e-61df12c19072\") " pod="test-ns-feast/postgres-864846d4f5-qcpsz" Jul 30 06:59:29.565866 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.565727 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6v88\" (UniqueName: \"kubernetes.io/projected/bac18c68-88dc-4e93-8f7e-61df12c19072-kube-api-access-v6v88\") pod \"postgres-864846d4f5-qcpsz\" (UID: \"bac18c68-88dc-4e93-8f7e-61df12c19072\") " pod="test-ns-feast/postgres-864846d4f5-qcpsz" Jul 30 06:59:29.566077 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.566041 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/bac18c68-88dc-4e93-8f7e-61df12c19072-postgresdata\") pod \"postgres-864846d4f5-qcpsz\" (UID: \"bac18c68-88dc-4e93-8f7e-61df12c19072\") " pod="test-ns-feast/postgres-864846d4f5-qcpsz" Jul 30 06:59:29.576481 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.576419 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6v88\" (UniqueName: \"kubernetes.io/projected/bac18c68-88dc-4e93-8f7e-61df12c19072-kube-api-access-v6v88\") pod \"postgres-864846d4f5-qcpsz\" (UID: \"bac18c68-88dc-4e93-8f7e-61df12c19072\") " pod="test-ns-feast/postgres-864846d4f5-qcpsz" Jul 30 06:59:29.654412 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.654375 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-qcpsz" Jul 30 06:59:29.666624 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.666598 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59srp\" (UniqueName: \"kubernetes.io/projected/899c8706-8461-484e-b25d-b41b26f5c38d-kube-api-access-59srp\") pod \"redis-69bbd798b6-dczqb\" (UID: \"899c8706-8461-484e-b25d-b41b26f5c38d\") " pod="test-ns-feast/redis-69bbd798b6-dczqb" Jul 30 06:59:29.681477 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.681437 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59srp\" (UniqueName: \"kubernetes.io/projected/899c8706-8461-484e-b25d-b41b26f5c38d-kube-api-access-59srp\") pod \"redis-69bbd798b6-dczqb\" (UID: \"899c8706-8461-484e-b25d-b41b26f5c38d\") " pod="test-ns-feast/redis-69bbd798b6-dczqb" Jul 30 06:59:29.734459 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.734153 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-dczqb" Jul 30 06:59:29.799376 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.799350 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-qcpsz"] Jul 30 06:59:29.802643 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:59:29.802620 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbac18c68_88dc_4e93_8f7e_61df12c19072.slice/crio-e9f997bd56324b04751c25143af6b14539df9caba82c8695237b718b80f3017a WatchSource:0}: Error finding container e9f997bd56324b04751c25143af6b14539df9caba82c8695237b718b80f3017a: Status 404 returned error can't find the container with id e9f997bd56324b04751c25143af6b14539df9caba82c8695237b718b80f3017a Jul 30 06:59:29.804903 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.804885 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 06:59:29.881072 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:29.881041 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-dczqb"] Jul 30 06:59:29.884451 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:59:29.884421 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod899c8706_8461_484e_b25d_b41b26f5c38d.slice/crio-7b370dfdda8a5694cfd83d0f97f0a17a81d091846924fd6ecc3d11aa30b8c17d WatchSource:0}: Error finding container 7b370dfdda8a5694cfd83d0f97f0a17a81d091846924fd6ecc3d11aa30b8c17d: Status 404 returned error can't find the container with id 7b370dfdda8a5694cfd83d0f97f0a17a81d091846924fd6ecc3d11aa30b8c17d Jul 30 06:59:30.790850 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:30.790816 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-dczqb" event={"ID":"899c8706-8461-484e-b25d-b41b26f5c38d","Type":"ContainerStarted","Data":"7b370dfdda8a5694cfd83d0f97f0a17a81d091846924fd6ecc3d11aa30b8c17d"} Jul 30 06:59:30.792077 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:30.792042 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-qcpsz" event={"ID":"bac18c68-88dc-4e93-8f7e-61df12c19072","Type":"ContainerStarted","Data":"e9f997bd56324b04751c25143af6b14539df9caba82c8695237b718b80f3017a"} Jul 30 06:59:32.802202 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:32.802165 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-dczqb" event={"ID":"899c8706-8461-484e-b25d-b41b26f5c38d","Type":"ContainerStarted","Data":"b18cad476c94a8d69b2a3cb9ac01bbd9f70aba7ebb3114ef215d3fb5d0bf8a97"} Jul 30 06:59:32.825789 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:32.825741 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-dczqb" podStartSLOduration=1.932152055 podStartE2EDuration="3.825722458s" totalImagesPullingTime="1.893570403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:59:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:59:32.825071237 +0000 UTC m=+934.182075398" watchObservedRunningTime="2026-07-30 06:59:32.825722458 +0000 UTC m=+934.182726597" Jul 30 06:59:35.815020 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:35.814984 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-qcpsz" event={"ID":"bac18c68-88dc-4e93-8f7e-61df12c19072","Type":"ContainerStarted","Data":"fbd152f40124493d302b0ed0d23a924224319ef54eca5f84307c7c46227d6034"} Jul 30 06:59:35.831487 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:35.831437 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-qcpsz" podStartSLOduration=1.80354243 podStartE2EDuration="6.831420708s" totalImagesPullingTime="5.027878278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:59:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:59:35.830402965 +0000 UTC m=+937.187407127" watchObservedRunningTime="2026-07-30 06:59:35.831420708 +0000 UTC m=+937.188424871" Jul 30 06:59:39.943052 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:39.943020 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc"] Jul 30 06:59:39.949810 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:39.949792 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:39.952694 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:39.952672 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Jul 30 06:59:39.952798 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:39.952676 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Jul 30 06:59:39.952798 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:39.952728 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Jul 30 06:59:39.953070 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:39.953049 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-n9gh9\"" Jul 30 06:59:39.953175 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:39.953052 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Jul 30 06:59:39.958040 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:39.958015 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc"] Jul 30 06:59:40.061600 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.061564 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-online-tls\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.061786 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.061612 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-offline-tls\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.061786 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.061712 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phbrj\" (UniqueName: \"kubernetes.io/projected/781ab876-e8b6-4e40-9a13-69cc595ec4cb-kube-api-access-phbrj\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.061786 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.061746 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-registry-tls\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.061786 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.061768 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/781ab876-e8b6-4e40-9a13-69cc595ec4cb-feast-data\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.162582 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.162540 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-online-tls\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.162804 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.162590 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-offline-tls\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.162804 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.162637 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phbrj\" (UniqueName: \"kubernetes.io/projected/781ab876-e8b6-4e40-9a13-69cc595ec4cb-kube-api-access-phbrj\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.162804 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.162696 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-registry-tls\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.162804 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.162718 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/781ab876-e8b6-4e40-9a13-69cc595ec4cb-feast-data\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.163014 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:59:40.162811 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Jul 30 06:59:40.163014 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:59:40.162892 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-registry-tls podName:781ab876-e8b6-4e40-9a13-69cc595ec4cb nodeName:}" failed. No retries permitted until 2026-07-30 06:59:40.662873042 +0000 UTC m=+942.019877184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-registry-tls") pod "feast-credit-scoring-6c6448844b-cc8sc" (UID: "781ab876-e8b6-4e40-9a13-69cc595ec4cb") : secret "feast-credit-scoring-registry-tls" not found Jul 30 06:59:40.163014 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:59:40.162809 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-offline-tls: secret "feast-credit-scoring-offline-tls" not found Jul 30 06:59:40.163014 ip-10-0-143-185 kubenswrapper[2575]: E0730 06:59:40.162989 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-offline-tls podName:781ab876-e8b6-4e40-9a13-69cc595ec4cb nodeName:}" failed. No retries permitted until 2026-07-30 06:59:40.662970065 +0000 UTC m=+942.019974215 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-offline-tls") pod "feast-credit-scoring-6c6448844b-cc8sc" (UID: "781ab876-e8b6-4e40-9a13-69cc595ec4cb") : secret "feast-credit-scoring-offline-tls" not found Jul 30 06:59:40.163200 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.163110 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/781ab876-e8b6-4e40-9a13-69cc595ec4cb-feast-data\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.165312 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.165289 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-online-tls\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.173101 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.173071 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phbrj\" (UniqueName: \"kubernetes.io/projected/781ab876-e8b6-4e40-9a13-69cc595ec4cb-kube-api-access-phbrj\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.666747 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.666703 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-offline-tls\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.666747 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.666756 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-registry-tls\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.669299 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.669270 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-offline-tls\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.669438 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.669409 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-registry-tls\") pod \"feast-credit-scoring-6c6448844b-cc8sc\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:40.861157 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:40.861119 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:41.034624 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:41.034535 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc"] Jul 30 06:59:41.037377 ip-10-0-143-185 kubenswrapper[2575]: W0730 06:59:41.037352 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod781ab876_e8b6_4e40_9a13_69cc595ec4cb.slice/crio-4e1d10be43b6bceb919f00837b781554689d711f464c8661b4257ab588f5aa0b WatchSource:0}: Error finding container 4e1d10be43b6bceb919f00837b781554689d711f464c8661b4257ab588f5aa0b: Status 404 returned error can't find the container with id 4e1d10be43b6bceb919f00837b781554689d711f464c8661b4257ab588f5aa0b Jul 30 06:59:41.841071 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:41.841031 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" event={"ID":"781ab876-e8b6-4e40-9a13-69cc595ec4cb","Type":"ContainerStarted","Data":"f1e70380d5d251aaebdb59bad66293a9cf1d6bbbb9c4922d84dcac30baa5bcba"} Jul 30 06:59:41.841269 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:41.841098 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" event={"ID":"781ab876-e8b6-4e40-9a13-69cc595ec4cb","Type":"ContainerStarted","Data":"4e1d10be43b6bceb919f00837b781554689d711f464c8661b4257ab588f5aa0b"} Jul 30 06:59:42.846518 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:42.846438 2575 generic.go:356] "Generic (PLEG): container finished" podID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerID="f1e70380d5d251aaebdb59bad66293a9cf1d6bbbb9c4922d84dcac30baa5bcba" exitCode=0 Jul 30 06:59:42.846892 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:42.846518 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" event={"ID":"781ab876-e8b6-4e40-9a13-69cc595ec4cb","Type":"ContainerDied","Data":"f1e70380d5d251aaebdb59bad66293a9cf1d6bbbb9c4922d84dcac30baa5bcba"} Jul 30 06:59:43.852394 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:43.852361 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" event={"ID":"781ab876-e8b6-4e40-9a13-69cc595ec4cb","Type":"ContainerStarted","Data":"7902daee5b6487ac79e666c08b6ab2518ff5d8de3d7d3065f47259e760bf4d3c"} Jul 30 06:59:49.881594 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:49.881557 2575 generic.go:356] "Generic (PLEG): container finished" podID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerID="7902daee5b6487ac79e666c08b6ab2518ff5d8de3d7d3065f47259e760bf4d3c" exitCode=0 Jul 30 06:59:49.881953 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:49.881607 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" event={"ID":"781ab876-e8b6-4e40-9a13-69cc595ec4cb","Type":"ContainerDied","Data":"7902daee5b6487ac79e666c08b6ab2518ff5d8de3d7d3065f47259e760bf4d3c"} Jul 30 06:59:50.890158 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:50.890115 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" event={"ID":"781ab876-e8b6-4e40-9a13-69cc595ec4cb","Type":"ContainerStarted","Data":"c112b0b342be7abbb618756f659f2347c8fa96da45c2757ce12095ca9969952f"} Jul 30 06:59:50.890763 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:50.890722 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" event={"ID":"781ab876-e8b6-4e40-9a13-69cc595ec4cb","Type":"ContainerStarted","Data":"ac958c065794ee0b2c415e40833905d144a6ec035f16cf3e4b149dd5c2d9b382"} Jul 30 06:59:50.890904 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:50.890770 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" event={"ID":"781ab876-e8b6-4e40-9a13-69cc595ec4cb","Type":"ContainerStarted","Data":"4f495b8d198f9f2133fbcea2e63029a6427fcc9548907915857c41e86845522c"} Jul 30 06:59:52.862198 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:52.862153 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:52.862929 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:52.862894 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:52.863054 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:52.862941 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:52.864077 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:52.864040 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.31:8816: connect: connection refused" Jul 30 06:59:52.864193 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:52.864078 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Jul 30 06:59:52.864329 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:52.864299 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.31:6571: connect: connection refused" Jul 30 06:59:55.862020 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:55.861978 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.31:8816: connect: connection refused" Jul 30 06:59:55.862775 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:55.861986 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.31:6571: connect: connection refused" Jul 30 06:59:56.313864 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:56.313827 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:56.314407 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:56.314380 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:56.317807 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:56.317780 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:56.338501 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:56.338438 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" podStartSLOduration=17.338405689 podStartE2EDuration="17.338405689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 06:59:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 06:59:50.924380195 +0000 UTC m=+952.281384358" watchObservedRunningTime="2026-07-30 06:59:56.338405689 +0000 UTC m=+957.695409851" Jul 30 06:59:58.861918 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:58.861867 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.31:6571: connect: connection refused" Jul 30 06:59:58.862424 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:58.862393 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:58.932454 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:58.932418 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 06:59:58.932814 ip-10-0-143-185 kubenswrapper[2575]: I0730 06:59:58.932794 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 07:00:00.861262 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:00:00.861228 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 07:00:01.867776 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:00:01.867732 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 07:00:01.945259 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:00:01.945230 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 07:01:08.160858 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.160822 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc"] Jul 30 07:01:08.161397 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.161085 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="registry" containerID="cri-o://4f495b8d198f9f2133fbcea2e63029a6427fcc9548907915857c41e86845522c" gracePeriod=30 Jul 30 07:01:08.161397 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.161130 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="offline" containerID="cri-o://c112b0b342be7abbb618756f659f2347c8fa96da45c2757ce12095ca9969952f" gracePeriod=30 Jul 30 07:01:08.161397 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.161149 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="online" containerID="cri-o://ac958c065794ee0b2c415e40833905d144a6ec035f16cf3e4b149dd5c2d9b382" gracePeriod=30 Jul 30 07:01:08.177815 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.177787 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-qcpsz"] Jul 30 07:01:08.177953 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.177934 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-qcpsz" podUID="bac18c68-88dc-4e93-8f7e-61df12c19072" containerName="postgres" containerID="cri-o://fbd152f40124493d302b0ed0d23a924224319ef54eca5f84307c7c46227d6034" gracePeriod=30 Jul 30 07:01:08.188542 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.188514 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-dczqb"] Jul 30 07:01:08.188725 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.188708 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-dczqb" podUID="899c8706-8461-484e-b25d-b41b26f5c38d" containerName="redis" containerID="cri-o://b18cad476c94a8d69b2a3cb9ac01bbd9f70aba7ebb3114ef215d3fb5d0bf8a97" gracePeriod=30 Jul 30 07:01:08.457180 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.457148 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-qcpsz" Jul 30 07:01:08.460093 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.460077 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-dczqb" Jul 30 07:01:08.499763 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.494872 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-59srp\" (UniqueName: \"kubernetes.io/projected/899c8706-8461-484e-b25d-b41b26f5c38d-kube-api-access-59srp\") pod \"899c8706-8461-484e-b25d-b41b26f5c38d\" (UID: \"899c8706-8461-484e-b25d-b41b26f5c38d\") " Jul 30 07:01:08.499763 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.494937 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6v88\" (UniqueName: \"kubernetes.io/projected/bac18c68-88dc-4e93-8f7e-61df12c19072-kube-api-access-v6v88\") pod \"bac18c68-88dc-4e93-8f7e-61df12c19072\" (UID: \"bac18c68-88dc-4e93-8f7e-61df12c19072\") " Jul 30 07:01:08.499763 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.494981 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/bac18c68-88dc-4e93-8f7e-61df12c19072-postgresdata\") pod \"bac18c68-88dc-4e93-8f7e-61df12c19072\" (UID: \"bac18c68-88dc-4e93-8f7e-61df12c19072\") " Jul 30 07:01:08.502324 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.502273 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bac18c68-88dc-4e93-8f7e-61df12c19072-kube-api-access-v6v88" (OuterVolumeSpecName: "kube-api-access-v6v88") pod "bac18c68-88dc-4e93-8f7e-61df12c19072" (UID: "bac18c68-88dc-4e93-8f7e-61df12c19072"). InnerVolumeSpecName "kube-api-access-v6v88". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 07:01:08.502324 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.502302 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/899c8706-8461-484e-b25d-b41b26f5c38d-kube-api-access-59srp" (OuterVolumeSpecName: "kube-api-access-59srp") pod "899c8706-8461-484e-b25d-b41b26f5c38d" (UID: "899c8706-8461-484e-b25d-b41b26f5c38d"). InnerVolumeSpecName "kube-api-access-59srp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 07:01:08.533850 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.533810 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac18c68-88dc-4e93-8f7e-61df12c19072-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "bac18c68-88dc-4e93-8f7e-61df12c19072" (UID: "bac18c68-88dc-4e93-8f7e-61df12c19072"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 07:01:08.596565 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.596529 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59srp\" (UniqueName: \"kubernetes.io/projected/899c8706-8461-484e-b25d-b41b26f5c38d-kube-api-access-59srp\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:01:08.596565 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.596557 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6v88\" (UniqueName: \"kubernetes.io/projected/bac18c68-88dc-4e93-8f7e-61df12c19072-kube-api-access-v6v88\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:01:08.596565 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:08.596569 2575 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/bac18c68-88dc-4e93-8f7e-61df12c19072-postgresdata\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:01:09.228391 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.228359 2575 generic.go:356] "Generic (PLEG): container finished" podID="bac18c68-88dc-4e93-8f7e-61df12c19072" containerID="fbd152f40124493d302b0ed0d23a924224319ef54eca5f84307c7c46227d6034" exitCode=0 Jul 30 07:01:09.228787 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.228424 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-qcpsz" Jul 30 07:01:09.228787 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.228450 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-qcpsz" event={"ID":"bac18c68-88dc-4e93-8f7e-61df12c19072","Type":"ContainerDied","Data":"fbd152f40124493d302b0ed0d23a924224319ef54eca5f84307c7c46227d6034"} Jul 30 07:01:09.228787 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.228499 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-qcpsz" event={"ID":"bac18c68-88dc-4e93-8f7e-61df12c19072","Type":"ContainerDied","Data":"e9f997bd56324b04751c25143af6b14539df9caba82c8695237b718b80f3017a"} Jul 30 07:01:09.228787 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.228522 2575 scope.go:117] "RemoveContainer" containerID="fbd152f40124493d302b0ed0d23a924224319ef54eca5f84307c7c46227d6034" Jul 30 07:01:09.231616 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.231594 2575 generic.go:356] "Generic (PLEG): container finished" podID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerID="ac958c065794ee0b2c415e40833905d144a6ec035f16cf3e4b149dd5c2d9b382" exitCode=0 Jul 30 07:01:09.231762 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.231672 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" event={"ID":"781ab876-e8b6-4e40-9a13-69cc595ec4cb","Type":"ContainerDied","Data":"ac958c065794ee0b2c415e40833905d144a6ec035f16cf3e4b149dd5c2d9b382"} Jul 30 07:01:09.232995 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.232971 2575 generic.go:356] "Generic (PLEG): container finished" podID="899c8706-8461-484e-b25d-b41b26f5c38d" containerID="b18cad476c94a8d69b2a3cb9ac01bbd9f70aba7ebb3114ef215d3fb5d0bf8a97" exitCode=0 Jul 30 07:01:09.233088 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.233026 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-dczqb" Jul 30 07:01:09.233133 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.233023 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-dczqb" event={"ID":"899c8706-8461-484e-b25d-b41b26f5c38d","Type":"ContainerDied","Data":"b18cad476c94a8d69b2a3cb9ac01bbd9f70aba7ebb3114ef215d3fb5d0bf8a97"} Jul 30 07:01:09.233168 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.233135 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-dczqb" event={"ID":"899c8706-8461-484e-b25d-b41b26f5c38d","Type":"ContainerDied","Data":"7b370dfdda8a5694cfd83d0f97f0a17a81d091846924fd6ecc3d11aa30b8c17d"} Jul 30 07:01:09.238844 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.238766 2575 scope.go:117] "RemoveContainer" containerID="fbd152f40124493d302b0ed0d23a924224319ef54eca5f84307c7c46227d6034" Jul 30 07:01:09.239420 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:01:09.239283 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbd152f40124493d302b0ed0d23a924224319ef54eca5f84307c7c46227d6034\": container with ID starting with fbd152f40124493d302b0ed0d23a924224319ef54eca5f84307c7c46227d6034 not found: ID does not exist" containerID="fbd152f40124493d302b0ed0d23a924224319ef54eca5f84307c7c46227d6034" Jul 30 07:01:09.239420 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.239315 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbd152f40124493d302b0ed0d23a924224319ef54eca5f84307c7c46227d6034"} err="failed to get container status \"fbd152f40124493d302b0ed0d23a924224319ef54eca5f84307c7c46227d6034\": rpc error: code = NotFound desc = could not find container \"fbd152f40124493d302b0ed0d23a924224319ef54eca5f84307c7c46227d6034\": container with ID starting with fbd152f40124493d302b0ed0d23a924224319ef54eca5f84307c7c46227d6034 not found: ID does not exist" Jul 30 07:01:09.239420 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.239337 2575 scope.go:117] "RemoveContainer" containerID="b18cad476c94a8d69b2a3cb9ac01bbd9f70aba7ebb3114ef215d3fb5d0bf8a97" Jul 30 07:01:09.244610 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.244590 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-qcpsz"] Jul 30 07:01:09.248003 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.247986 2575 scope.go:117] "RemoveContainer" containerID="b18cad476c94a8d69b2a3cb9ac01bbd9f70aba7ebb3114ef215d3fb5d0bf8a97" Jul 30 07:01:09.248247 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.248231 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-qcpsz"] Jul 30 07:01:09.248303 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:01:09.248246 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b18cad476c94a8d69b2a3cb9ac01bbd9f70aba7ebb3114ef215d3fb5d0bf8a97\": container with ID starting with b18cad476c94a8d69b2a3cb9ac01bbd9f70aba7ebb3114ef215d3fb5d0bf8a97 not found: ID does not exist" containerID="b18cad476c94a8d69b2a3cb9ac01bbd9f70aba7ebb3114ef215d3fb5d0bf8a97" Jul 30 07:01:09.248303 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.248269 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b18cad476c94a8d69b2a3cb9ac01bbd9f70aba7ebb3114ef215d3fb5d0bf8a97"} err="failed to get container status \"b18cad476c94a8d69b2a3cb9ac01bbd9f70aba7ebb3114ef215d3fb5d0bf8a97\": rpc error: code = NotFound desc = could not find container \"b18cad476c94a8d69b2a3cb9ac01bbd9f70aba7ebb3114ef215d3fb5d0bf8a97\": container with ID starting with b18cad476c94a8d69b2a3cb9ac01bbd9f70aba7ebb3114ef215d3fb5d0bf8a97 not found: ID does not exist" Jul 30 07:01:09.259494 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.259472 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-dczqb"] Jul 30 07:01:09.263132 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:09.263113 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-dczqb"] Jul 30 07:01:11.175868 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:11.175835 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="899c8706-8461-484e-b25d-b41b26f5c38d" path="/var/lib/kubelet/pods/899c8706-8461-484e-b25d-b41b26f5c38d/volumes" Jul 30 07:01:11.176230 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:11.176176 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bac18c68-88dc-4e93-8f7e-61df12c19072" path="/var/lib/kubelet/pods/bac18c68-88dc-4e93-8f7e-61df12c19072/volumes" Jul 30 07:01:16.314468 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:16.314431 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Jul 30 07:01:26.314589 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:26.314548 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Jul 30 07:01:36.314871 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:36.314825 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Jul 30 07:01:36.315351 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:36.314953 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 07:01:38.362168 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:38.362137 2575 generic.go:356] "Generic (PLEG): container finished" podID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerID="c112b0b342be7abbb618756f659f2347c8fa96da45c2757ce12095ca9969952f" exitCode=137 Jul 30 07:01:38.362168 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:38.362163 2575 generic.go:356] "Generic (PLEG): container finished" podID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerID="4f495b8d198f9f2133fbcea2e63029a6427fcc9548907915857c41e86845522c" exitCode=137 Jul 30 07:01:38.362669 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:38.362203 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" event={"ID":"781ab876-e8b6-4e40-9a13-69cc595ec4cb","Type":"ContainerDied","Data":"c112b0b342be7abbb618756f659f2347c8fa96da45c2757ce12095ca9969952f"} Jul 30 07:01:38.362669 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:38.362243 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" event={"ID":"781ab876-e8b6-4e40-9a13-69cc595ec4cb","Type":"ContainerDied","Data":"4f495b8d198f9f2133fbcea2e63029a6427fcc9548907915857c41e86845522c"} Jul 30 07:01:38.817328 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:38.817303 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 07:01:38.939916 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:38.939839 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phbrj\" (UniqueName: \"kubernetes.io/projected/781ab876-e8b6-4e40-9a13-69cc595ec4cb-kube-api-access-phbrj\") pod \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " Jul 30 07:01:38.940162 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:38.939922 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-offline-tls\") pod \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " Jul 30 07:01:38.940162 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:38.939954 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-online-tls\") pod \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " Jul 30 07:01:38.940162 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:38.939983 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/781ab876-e8b6-4e40-9a13-69cc595ec4cb-feast-data\") pod \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " Jul 30 07:01:38.940162 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:38.940004 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-registry-tls\") pod \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\" (UID: \"781ab876-e8b6-4e40-9a13-69cc595ec4cb\") " Jul 30 07:01:38.942186 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:38.942152 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/781ab876-e8b6-4e40-9a13-69cc595ec4cb-kube-api-access-phbrj" (OuterVolumeSpecName: "kube-api-access-phbrj") pod "781ab876-e8b6-4e40-9a13-69cc595ec4cb" (UID: "781ab876-e8b6-4e40-9a13-69cc595ec4cb"). InnerVolumeSpecName "kube-api-access-phbrj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 07:01:38.942572 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:38.942544 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "781ab876-e8b6-4e40-9a13-69cc595ec4cb" (UID: "781ab876-e8b6-4e40-9a13-69cc595ec4cb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:01:38.942778 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:38.942635 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "781ab876-e8b6-4e40-9a13-69cc595ec4cb" (UID: "781ab876-e8b6-4e40-9a13-69cc595ec4cb"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:01:38.942778 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:38.942647 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-online-tls" (OuterVolumeSpecName: "online-tls") pod "781ab876-e8b6-4e40-9a13-69cc595ec4cb" (UID: "781ab876-e8b6-4e40-9a13-69cc595ec4cb"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:01:38.949275 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:38.949248 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/781ab876-e8b6-4e40-9a13-69cc595ec4cb-feast-data" (OuterVolumeSpecName: "feast-data") pod "781ab876-e8b6-4e40-9a13-69cc595ec4cb" (UID: "781ab876-e8b6-4e40-9a13-69cc595ec4cb"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 07:01:39.040832 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:39.040807 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-offline-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:01:39.040832 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:39.040830 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-online-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:01:39.040832 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:39.040839 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/781ab876-e8b6-4e40-9a13-69cc595ec4cb-feast-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:01:39.041028 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:39.040846 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/781ab876-e8b6-4e40-9a13-69cc595ec4cb-registry-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:01:39.041028 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:39.040855 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phbrj\" (UniqueName: \"kubernetes.io/projected/781ab876-e8b6-4e40-9a13-69cc595ec4cb-kube-api-access-phbrj\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:01:39.371779 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:39.371690 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" event={"ID":"781ab876-e8b6-4e40-9a13-69cc595ec4cb","Type":"ContainerDied","Data":"4e1d10be43b6bceb919f00837b781554689d711f464c8661b4257ab588f5aa0b"} Jul 30 07:01:39.371779 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:39.371741 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc" Jul 30 07:01:39.371779 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:39.371757 2575 scope.go:117] "RemoveContainer" containerID="c112b0b342be7abbb618756f659f2347c8fa96da45c2757ce12095ca9969952f" Jul 30 07:01:39.380376 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:39.380355 2575 scope.go:117] "RemoveContainer" containerID="ac958c065794ee0b2c415e40833905d144a6ec035f16cf3e4b149dd5c2d9b382" Jul 30 07:01:39.388823 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:39.388802 2575 scope.go:117] "RemoveContainer" containerID="4f495b8d198f9f2133fbcea2e63029a6427fcc9548907915857c41e86845522c" Jul 30 07:01:39.392722 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:39.392702 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc"] Jul 30 07:01:39.395769 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:39.395749 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-6c6448844b-cc8sc"] Jul 30 07:01:39.398775 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:39.398760 2575 scope.go:117] "RemoveContainer" containerID="7902daee5b6487ac79e666c08b6ab2518ff5d8de3d7d3065f47259e760bf4d3c" Jul 30 07:01:39.406060 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:39.406043 2575 scope.go:117] "RemoveContainer" containerID="f1e70380d5d251aaebdb59bad66293a9cf1d6bbbb9c4922d84dcac30baa5bcba" Jul 30 07:01:41.179855 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:01:41.177504 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" path="/var/lib/kubelet/pods/781ab876-e8b6-4e40-9a13-69cc595ec4cb/volumes" Jul 30 07:03:59.289169 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:03:59.289058 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/1.log" Jul 30 07:03:59.292135 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:03:59.291186 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/1.log" Jul 30 07:03:59.296407 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:03:59.296385 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 07:03:59.298226 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:03:59.298209 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 07:04:18.072441 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072405 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-8bmr9"] Jul 30 07:04:18.072834 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072718 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="registry" Jul 30 07:04:18.072834 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072731 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="registry" Jul 30 07:04:18.072834 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072743 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bac18c68-88dc-4e93-8f7e-61df12c19072" containerName="postgres" Jul 30 07:04:18.072834 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072748 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac18c68-88dc-4e93-8f7e-61df12c19072" containerName="postgres" Jul 30 07:04:18.072834 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072755 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="899c8706-8461-484e-b25d-b41b26f5c38d" containerName="redis" Jul 30 07:04:18.072834 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072761 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="899c8706-8461-484e-b25d-b41b26f5c38d" containerName="redis" Jul 30 07:04:18.072834 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072772 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="feast-init" Jul 30 07:04:18.072834 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072778 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="feast-init" Jul 30 07:04:18.072834 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072785 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="online" Jul 30 07:04:18.072834 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072790 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="online" Jul 30 07:04:18.072834 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072801 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="feast-apply" Jul 30 07:04:18.072834 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072806 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="feast-apply" Jul 30 07:04:18.072834 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072814 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="offline" Jul 30 07:04:18.072834 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072819 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="offline" Jul 30 07:04:18.073329 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072860 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="899c8706-8461-484e-b25d-b41b26f5c38d" containerName="redis" Jul 30 07:04:18.073329 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072868 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="offline" Jul 30 07:04:18.073329 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072873 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="online" Jul 30 07:04:18.073329 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072879 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="781ab876-e8b6-4e40-9a13-69cc595ec4cb" containerName="registry" Jul 30 07:04:18.073329 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.072885 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="bac18c68-88dc-4e93-8f7e-61df12c19072" containerName="postgres" Jul 30 07:04:18.075938 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.075921 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-8bmr9" Jul 30 07:04:18.078845 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.078820 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Jul 30 07:04:18.078967 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.078842 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-8tq6f\"" Jul 30 07:04:18.078967 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.078861 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Jul 30 07:04:18.094781 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.094747 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-8bmr9"] Jul 30 07:04:18.192340 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.192312 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nmcb\" (UniqueName: \"kubernetes.io/projected/87273ba1-8337-4b11-82e4-171df0c13e17-kube-api-access-9nmcb\") pod \"redis-69bbd798b6-8bmr9\" (UID: \"87273ba1-8337-4b11-82e4-171df0c13e17\") " pod="test-ns-feast-rest/redis-69bbd798b6-8bmr9" Jul 30 07:04:18.293047 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.293007 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nmcb\" (UniqueName: \"kubernetes.io/projected/87273ba1-8337-4b11-82e4-171df0c13e17-kube-api-access-9nmcb\") pod \"redis-69bbd798b6-8bmr9\" (UID: \"87273ba1-8337-4b11-82e4-171df0c13e17\") " pod="test-ns-feast-rest/redis-69bbd798b6-8bmr9" Jul 30 07:04:18.302554 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.302530 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nmcb\" (UniqueName: \"kubernetes.io/projected/87273ba1-8337-4b11-82e4-171df0c13e17-kube-api-access-9nmcb\") pod \"redis-69bbd798b6-8bmr9\" (UID: \"87273ba1-8337-4b11-82e4-171df0c13e17\") " pod="test-ns-feast-rest/redis-69bbd798b6-8bmr9" Jul 30 07:04:18.402648 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.402615 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-8bmr9" Jul 30 07:04:18.532540 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.532510 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-8bmr9"] Jul 30 07:04:18.535179 ip-10-0-143-185 kubenswrapper[2575]: W0730 07:04:18.535141 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87273ba1_8337_4b11_82e4_171df0c13e17.slice/crio-cd551d43cbd079bb13dbbe88ea621d4c3ec8293796733d9d5645f04c781401ee WatchSource:0}: Error finding container cd551d43cbd079bb13dbbe88ea621d4c3ec8293796733d9d5645f04c781401ee: Status 404 returned error can't find the container with id cd551d43cbd079bb13dbbe88ea621d4c3ec8293796733d9d5645f04c781401ee Jul 30 07:04:18.980725 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.980693 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-8bmr9" event={"ID":"87273ba1-8337-4b11-82e4-171df0c13e17","Type":"ContainerStarted","Data":"ff17c24a6f3d887910cde52f4903a6be7b49284ead127a7a0a8a39cf9f31a03e"} Jul 30 07:04:18.980895 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.980732 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-8bmr9" event={"ID":"87273ba1-8337-4b11-82e4-171df0c13e17","Type":"ContainerStarted","Data":"cd551d43cbd079bb13dbbe88ea621d4c3ec8293796733d9d5645f04c781401ee"} Jul 30 07:04:18.997153 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:18.997105 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-8bmr9" podStartSLOduration=0.997089043 podStartE2EDuration="997.089043ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 07:04:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 07:04:18.99613103 +0000 UTC m=+1220.353135192" watchObservedRunningTime="2026-07-30 07:04:18.997089043 +0000 UTC m=+1220.354093205" Jul 30 07:04:23.508040 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:23.508003 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-s5vz2"] Jul 30 07:04:23.511317 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:23.511302 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-s5vz2" Jul 30 07:04:23.513768 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:23.513744 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Jul 30 07:04:23.519407 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:23.519385 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-s5vz2"] Jul 30 07:04:23.530573 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:23.530551 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/feca6faf-a9fc-489f-80fe-e04807eb4f6d-postgresdata\") pod \"postgres-864846d4f5-s5vz2\" (UID: \"feca6faf-a9fc-489f-80fe-e04807eb4f6d\") " pod="test-ns-feast-rest/postgres-864846d4f5-s5vz2" Jul 30 07:04:23.530687 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:23.530629 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c4bn\" (UniqueName: \"kubernetes.io/projected/feca6faf-a9fc-489f-80fe-e04807eb4f6d-kube-api-access-9c4bn\") pod \"postgres-864846d4f5-s5vz2\" (UID: \"feca6faf-a9fc-489f-80fe-e04807eb4f6d\") " pod="test-ns-feast-rest/postgres-864846d4f5-s5vz2" Jul 30 07:04:23.631905 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:23.631868 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c4bn\" (UniqueName: \"kubernetes.io/projected/feca6faf-a9fc-489f-80fe-e04807eb4f6d-kube-api-access-9c4bn\") pod \"postgres-864846d4f5-s5vz2\" (UID: \"feca6faf-a9fc-489f-80fe-e04807eb4f6d\") " pod="test-ns-feast-rest/postgres-864846d4f5-s5vz2" Jul 30 07:04:23.632077 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:23.631963 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/feca6faf-a9fc-489f-80fe-e04807eb4f6d-postgresdata\") pod \"postgres-864846d4f5-s5vz2\" (UID: \"feca6faf-a9fc-489f-80fe-e04807eb4f6d\") " pod="test-ns-feast-rest/postgres-864846d4f5-s5vz2" Jul 30 07:04:23.632420 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:23.632399 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/feca6faf-a9fc-489f-80fe-e04807eb4f6d-postgresdata\") pod \"postgres-864846d4f5-s5vz2\" (UID: \"feca6faf-a9fc-489f-80fe-e04807eb4f6d\") " pod="test-ns-feast-rest/postgres-864846d4f5-s5vz2" Jul 30 07:04:23.640492 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:23.640475 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c4bn\" (UniqueName: \"kubernetes.io/projected/feca6faf-a9fc-489f-80fe-e04807eb4f6d-kube-api-access-9c4bn\") pod \"postgres-864846d4f5-s5vz2\" (UID: \"feca6faf-a9fc-489f-80fe-e04807eb4f6d\") " pod="test-ns-feast-rest/postgres-864846d4f5-s5vz2" Jul 30 07:04:23.820948 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:23.820863 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-s5vz2" Jul 30 07:04:23.951952 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:23.951923 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-s5vz2"] Jul 30 07:04:23.955528 ip-10-0-143-185 kubenswrapper[2575]: W0730 07:04:23.955489 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfeca6faf_a9fc_489f_80fe_e04807eb4f6d.slice/crio-ecdbbcd7b4339c1e1d6adb80b3382dfb2e58649f7887e493abfb123a05b108a1 WatchSource:0}: Error finding container ecdbbcd7b4339c1e1d6adb80b3382dfb2e58649f7887e493abfb123a05b108a1: Status 404 returned error can't find the container with id ecdbbcd7b4339c1e1d6adb80b3382dfb2e58649f7887e493abfb123a05b108a1 Jul 30 07:04:24.000862 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:24.000834 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-s5vz2" event={"ID":"feca6faf-a9fc-489f-80fe-e04807eb4f6d","Type":"ContainerStarted","Data":"ecdbbcd7b4339c1e1d6adb80b3382dfb2e58649f7887e493abfb123a05b108a1"} Jul 30 07:04:25.006394 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:25.006360 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-s5vz2" event={"ID":"feca6faf-a9fc-489f-80fe-e04807eb4f6d","Type":"ContainerStarted","Data":"17ef5388e3337b97b4bf57ecdc5ba261b898f416ea1f8bce3a0e264047ffdbcc"} Jul 30 07:04:25.024397 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:25.024352 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-s5vz2" podStartSLOduration=2.024336071 podStartE2EDuration="2.024336071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 07:04:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 07:04:25.023284052 +0000 UTC m=+1226.380288215" watchObservedRunningTime="2026-07-30 07:04:25.024336071 +0000 UTC m=+1226.381340233" Jul 30 07:04:29.089398 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.089357 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls"] Jul 30 07:04:29.092882 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.092860 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:29.100494 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.100474 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Jul 30 07:04:29.100617 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.100587 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Jul 30 07:04:29.100752 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.100734 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-ffksh\"" Jul 30 07:04:29.101310 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.101296 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Jul 30 07:04:29.111026 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.111001 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls"] Jul 30 07:04:29.171109 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.171075 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/36be7fcf-d199-4dd1-9083-2a708b5765e7-online-tls\") pod \"feast-credit-scoring-f7b5fcf5c-7xsls\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:29.171109 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.171113 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/36be7fcf-d199-4dd1-9083-2a708b5765e7-registry-tls\") pod \"feast-credit-scoring-f7b5fcf5c-7xsls\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:29.171371 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.171187 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kmr9\" (UniqueName: \"kubernetes.io/projected/36be7fcf-d199-4dd1-9083-2a708b5765e7-kube-api-access-4kmr9\") pod \"feast-credit-scoring-f7b5fcf5c-7xsls\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:29.171371 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.171234 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/36be7fcf-d199-4dd1-9083-2a708b5765e7-feast-data\") pod \"feast-credit-scoring-f7b5fcf5c-7xsls\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:29.271775 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.271747 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kmr9\" (UniqueName: \"kubernetes.io/projected/36be7fcf-d199-4dd1-9083-2a708b5765e7-kube-api-access-4kmr9\") pod \"feast-credit-scoring-f7b5fcf5c-7xsls\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:29.271949 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.271807 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/36be7fcf-d199-4dd1-9083-2a708b5765e7-feast-data\") pod \"feast-credit-scoring-f7b5fcf5c-7xsls\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:29.271949 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.271854 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/36be7fcf-d199-4dd1-9083-2a708b5765e7-online-tls\") pod \"feast-credit-scoring-f7b5fcf5c-7xsls\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:29.271949 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.271888 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/36be7fcf-d199-4dd1-9083-2a708b5765e7-registry-tls\") pod \"feast-credit-scoring-f7b5fcf5c-7xsls\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:29.272199 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:04:29.272012 2575 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Jul 30 07:04:29.272199 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:04:29.272081 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36be7fcf-d199-4dd1-9083-2a708b5765e7-registry-tls podName:36be7fcf-d199-4dd1-9083-2a708b5765e7 nodeName:}" failed. No retries permitted until 2026-07-30 07:04:29.772059974 +0000 UTC m=+1231.129064118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/36be7fcf-d199-4dd1-9083-2a708b5765e7-registry-tls") pod "feast-credit-scoring-f7b5fcf5c-7xsls" (UID: "36be7fcf-d199-4dd1-9083-2a708b5765e7") : secret "feast-credit-scoring-registry-tls" not found Jul 30 07:04:29.272314 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.272212 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/36be7fcf-d199-4dd1-9083-2a708b5765e7-feast-data\") pod \"feast-credit-scoring-f7b5fcf5c-7xsls\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:29.274403 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.274385 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/36be7fcf-d199-4dd1-9083-2a708b5765e7-online-tls\") pod \"feast-credit-scoring-f7b5fcf5c-7xsls\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:29.281542 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.281524 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kmr9\" (UniqueName: \"kubernetes.io/projected/36be7fcf-d199-4dd1-9083-2a708b5765e7-kube-api-access-4kmr9\") pod \"feast-credit-scoring-f7b5fcf5c-7xsls\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:29.774388 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.774354 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/36be7fcf-d199-4dd1-9083-2a708b5765e7-registry-tls\") pod \"feast-credit-scoring-f7b5fcf5c-7xsls\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:29.777026 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:29.777002 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/36be7fcf-d199-4dd1-9083-2a708b5765e7-registry-tls\") pod \"feast-credit-scoring-f7b5fcf5c-7xsls\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:30.002288 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:30.002235 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:30.141774 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:30.141748 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls"] Jul 30 07:04:30.144436 ip-10-0-143-185 kubenswrapper[2575]: W0730 07:04:30.144413 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36be7fcf_d199_4dd1_9083_2a708b5765e7.slice/crio-c809da6ac5fe6aa3f15a0cf6d4a07a0f02addf27b5d38717e5ce37470b83a57f WatchSource:0}: Error finding container c809da6ac5fe6aa3f15a0cf6d4a07a0f02addf27b5d38717e5ce37470b83a57f: Status 404 returned error can't find the container with id c809da6ac5fe6aa3f15a0cf6d4a07a0f02addf27b5d38717e5ce37470b83a57f Jul 30 07:04:31.033739 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:31.033700 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" event={"ID":"36be7fcf-d199-4dd1-9083-2a708b5765e7","Type":"ContainerStarted","Data":"524e4685755a3299cc3a035074085ee9bee4190b3c40b2af9436a58ebe453d9c"} Jul 30 07:04:31.033942 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:31.033763 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" event={"ID":"36be7fcf-d199-4dd1-9083-2a708b5765e7","Type":"ContainerStarted","Data":"c809da6ac5fe6aa3f15a0cf6d4a07a0f02addf27b5d38717e5ce37470b83a57f"} Jul 30 07:04:32.044169 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:32.044132 2575 generic.go:356] "Generic (PLEG): container finished" podID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerID="524e4685755a3299cc3a035074085ee9bee4190b3c40b2af9436a58ebe453d9c" exitCode=0 Jul 30 07:04:32.044546 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:32.044212 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" event={"ID":"36be7fcf-d199-4dd1-9083-2a708b5765e7","Type":"ContainerDied","Data":"524e4685755a3299cc3a035074085ee9bee4190b3c40b2af9436a58ebe453d9c"} Jul 30 07:04:33.049712 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:33.049676 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" event={"ID":"36be7fcf-d199-4dd1-9083-2a708b5765e7","Type":"ContainerStarted","Data":"607308c17e63697c843c6c0134b3d2b06da12f65108d14b4c448980d147a7df1"} Jul 30 07:04:41.090326 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:41.090291 2575 generic.go:356] "Generic (PLEG): container finished" podID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerID="607308c17e63697c843c6c0134b3d2b06da12f65108d14b4c448980d147a7df1" exitCode=0 Jul 30 07:04:41.090732 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:41.090349 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" event={"ID":"36be7fcf-d199-4dd1-9083-2a708b5765e7","Type":"ContainerDied","Data":"607308c17e63697c843c6c0134b3d2b06da12f65108d14b4c448980d147a7df1"} Jul 30 07:04:42.097227 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:42.097184 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" event={"ID":"36be7fcf-d199-4dd1-9083-2a708b5765e7","Type":"ContainerStarted","Data":"0397db14c39d2aa0c77b1799187192bccedcfffcba4e1207084fcf2d87fe5eec"} Jul 30 07:04:42.097750 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:42.097260 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" event={"ID":"36be7fcf-d199-4dd1-9083-2a708b5765e7","Type":"ContainerStarted","Data":"87cabcc106ae4ece0217010f62bd021afcbf83cc581c043eee9be0bd0b2092d6"} Jul 30 07:04:42.131446 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:42.131384 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" podStartSLOduration=13.131367321 podStartE2EDuration="13.131367321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 07:04:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 07:04:42.120671571 +0000 UTC m=+1243.477675731" watchObservedRunningTime="2026-07-30 07:04:42.131367321 +0000 UTC m=+1243.488371482" Jul 30 07:04:45.002628 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:45.002592 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:45.003108 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:45.002693 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:45.004115 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:45.004084 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.34:6571: connect: connection refused" Jul 30 07:04:45.004247 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:45.004123 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Jul 30 07:04:48.003091 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:48.003038 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.34:6571: connect: connection refused" Jul 30 07:04:48.008717 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:48.008693 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:48.009199 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:48.009183 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:48.012097 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:48.012080 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:50.003233 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:50.003143 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:51.003484 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:51.003435 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.34:6571: connect: connection refused" Jul 30 07:04:54.003769 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:54.003739 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:54.004474 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:54.004455 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:04:54.979990 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:54.979954 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r"] Jul 30 07:04:54.983216 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:54.983193 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:04:54.987878 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:54.986611 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Jul 30 07:04:54.987878 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:54.987063 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Jul 30 07:04:54.987878 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:54.987355 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-4mhpg\"" Jul 30 07:04:54.987878 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:54.987707 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Jul 30 07:04:54.998677 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:54.998226 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r"] Jul 30 07:04:55.065396 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:55.065354 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0bbf6246-9fe0-4d1d-9899-de59549688fd-online-tls\") pod \"feast-driver-ranking-7b5457bb66-r6d7r\" (UID: \"0bbf6246-9fe0-4d1d-9899-de59549688fd\") " pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:04:55.065901 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:55.065474 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jg76\" (UniqueName: \"kubernetes.io/projected/0bbf6246-9fe0-4d1d-9899-de59549688fd-kube-api-access-5jg76\") pod \"feast-driver-ranking-7b5457bb66-r6d7r\" (UID: \"0bbf6246-9fe0-4d1d-9899-de59549688fd\") " pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:04:55.065901 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:55.065534 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0bbf6246-9fe0-4d1d-9899-de59549688fd-registry-tls\") pod \"feast-driver-ranking-7b5457bb66-r6d7r\" (UID: \"0bbf6246-9fe0-4d1d-9899-de59549688fd\") " pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:04:55.065901 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:55.065596 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0bbf6246-9fe0-4d1d-9899-de59549688fd-feast-data\") pod \"feast-driver-ranking-7b5457bb66-r6d7r\" (UID: \"0bbf6246-9fe0-4d1d-9899-de59549688fd\") " pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:04:55.166807 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:55.166773 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jg76\" (UniqueName: \"kubernetes.io/projected/0bbf6246-9fe0-4d1d-9899-de59549688fd-kube-api-access-5jg76\") pod \"feast-driver-ranking-7b5457bb66-r6d7r\" (UID: \"0bbf6246-9fe0-4d1d-9899-de59549688fd\") " pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:04:55.166986 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:55.166826 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0bbf6246-9fe0-4d1d-9899-de59549688fd-registry-tls\") pod \"feast-driver-ranking-7b5457bb66-r6d7r\" (UID: \"0bbf6246-9fe0-4d1d-9899-de59549688fd\") " pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:04:55.166986 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:55.166872 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0bbf6246-9fe0-4d1d-9899-de59549688fd-feast-data\") pod \"feast-driver-ranking-7b5457bb66-r6d7r\" (UID: \"0bbf6246-9fe0-4d1d-9899-de59549688fd\") " pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:04:55.166986 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:55.166923 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0bbf6246-9fe0-4d1d-9899-de59549688fd-online-tls\") pod \"feast-driver-ranking-7b5457bb66-r6d7r\" (UID: \"0bbf6246-9fe0-4d1d-9899-de59549688fd\") " pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:04:55.167318 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:55.167293 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0bbf6246-9fe0-4d1d-9899-de59549688fd-feast-data\") pod \"feast-driver-ranking-7b5457bb66-r6d7r\" (UID: \"0bbf6246-9fe0-4d1d-9899-de59549688fd\") " pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:04:55.169491 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:55.169461 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0bbf6246-9fe0-4d1d-9899-de59549688fd-online-tls\") pod \"feast-driver-ranking-7b5457bb66-r6d7r\" (UID: \"0bbf6246-9fe0-4d1d-9899-de59549688fd\") " pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:04:55.169678 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:55.169635 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0bbf6246-9fe0-4d1d-9899-de59549688fd-registry-tls\") pod \"feast-driver-ranking-7b5457bb66-r6d7r\" (UID: \"0bbf6246-9fe0-4d1d-9899-de59549688fd\") " pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:04:55.176938 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:55.176911 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jg76\" (UniqueName: \"kubernetes.io/projected/0bbf6246-9fe0-4d1d-9899-de59549688fd-kube-api-access-5jg76\") pod \"feast-driver-ranking-7b5457bb66-r6d7r\" (UID: \"0bbf6246-9fe0-4d1d-9899-de59549688fd\") " pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:04:55.301797 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:55.301706 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:04:55.453903 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:55.453879 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r"] Jul 30 07:04:55.456274 ip-10-0-143-185 kubenswrapper[2575]: W0730 07:04:55.456238 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bbf6246_9fe0_4d1d_9899_de59549688fd.slice/crio-8cb1e209467ca16bcd0b278ceced883a228f1ebd73e100f9d9d9520cf14cbd7c WatchSource:0}: Error finding container 8cb1e209467ca16bcd0b278ceced883a228f1ebd73e100f9d9d9520cf14cbd7c: Status 404 returned error can't find the container with id 8cb1e209467ca16bcd0b278ceced883a228f1ebd73e100f9d9d9520cf14cbd7c Jul 30 07:04:56.161802 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:56.161765 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" event={"ID":"0bbf6246-9fe0-4d1d-9899-de59549688fd","Type":"ContainerStarted","Data":"31bb4830ca18b13303c5c4cc0013f80573d2c02e4a4dec519a82e7438abf090f"} Jul 30 07:04:56.162183 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:04:56.161818 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" event={"ID":"0bbf6246-9fe0-4d1d-9899-de59549688fd","Type":"ContainerStarted","Data":"8cb1e209467ca16bcd0b278ceced883a228f1ebd73e100f9d9d9520cf14cbd7c"} Jul 30 07:05:01.183154 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:01.183124 2575 generic.go:356] "Generic (PLEG): container finished" podID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerID="31bb4830ca18b13303c5c4cc0013f80573d2c02e4a4dec519a82e7438abf090f" exitCode=0 Jul 30 07:05:01.183597 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:01.183163 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" event={"ID":"0bbf6246-9fe0-4d1d-9899-de59549688fd","Type":"ContainerDied","Data":"31bb4830ca18b13303c5c4cc0013f80573d2c02e4a4dec519a82e7438abf090f"} Jul 30 07:05:02.188391 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:02.188359 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" event={"ID":"0bbf6246-9fe0-4d1d-9899-de59549688fd","Type":"ContainerStarted","Data":"87a8ea09f3bd37d29af9c46235eb9e28207514f889178e7dbf83d12b7cd1c85b"} Jul 30 07:05:08.215720 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:08.215627 2575 generic.go:356] "Generic (PLEG): container finished" podID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerID="87a8ea09f3bd37d29af9c46235eb9e28207514f889178e7dbf83d12b7cd1c85b" exitCode=0 Jul 30 07:05:08.215720 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:08.215696 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" event={"ID":"0bbf6246-9fe0-4d1d-9899-de59549688fd","Type":"ContainerDied","Data":"87a8ea09f3bd37d29af9c46235eb9e28207514f889178e7dbf83d12b7cd1c85b"} Jul 30 07:05:08.219249 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:08.219232 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Jul 30 07:05:09.222626 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:09.222592 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" event={"ID":"0bbf6246-9fe0-4d1d-9899-de59549688fd","Type":"ContainerStarted","Data":"23df4edaa55a9b450a123b27388643f92d21b7fd7f9f7b72991670cf69b4ef2d"} Jul 30 07:05:09.223015 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:09.222648 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" event={"ID":"0bbf6246-9fe0-4d1d-9899-de59549688fd","Type":"ContainerStarted","Data":"87df565c46543acdfdba9bec0357e4615d186b61b0bf8a3c5a52961e5d718bfa"} Jul 30 07:05:09.247541 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:09.247480 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" podStartSLOduration=15.247462661 podStartE2EDuration="15.247462661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 07:04:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 07:05:09.245394236 +0000 UTC m=+1270.602398411" watchObservedRunningTime="2026-07-30 07:05:09.247462661 +0000 UTC m=+1270.604466825" Jul 30 07:05:10.302362 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:10.302314 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:05:10.302763 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:10.302437 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:05:10.303702 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:10.303650 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused" Jul 30 07:05:10.303853 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:10.303722 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Jul 30 07:05:13.302231 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:13.302186 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused" Jul 30 07:05:13.302231 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:13.302205 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Jul 30 07:05:15.302744 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:15.302706 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:05:15.303210 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:15.302786 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:05:16.302676 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:16.302616 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused" Jul 30 07:05:16.308679 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:16.308642 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:05:16.312302 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:16.312278 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:05:19.303094 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:19.303051 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.35:6571: connect: connection refused" Jul 30 07:05:22.303470 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:22.303439 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:05:22.304164 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:05:22.304144 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:06:43.866632 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:43.866556 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4"] Jul 30 07:06:43.867135 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:43.866761 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4" podUID="cfc51ead-a2f4-465a-aa9d-7c3f075c06e2" containerName="manager" containerID="cri-o://2ccfe685ecde79f9dde6e87c821a9e96613b40d1374672df9271a5358fc7361d" gracePeriod=10 Jul 30 07:06:44.145775 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.145747 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4" Jul 30 07:06:44.199922 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.199895 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdf7l\" (UniqueName: \"kubernetes.io/projected/cfc51ead-a2f4-465a-aa9d-7c3f075c06e2-kube-api-access-tdf7l\") pod \"cfc51ead-a2f4-465a-aa9d-7c3f075c06e2\" (UID: \"cfc51ead-a2f4-465a-aa9d-7c3f075c06e2\") " Jul 30 07:06:44.202190 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.202155 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfc51ead-a2f4-465a-aa9d-7c3f075c06e2-kube-api-access-tdf7l" (OuterVolumeSpecName: "kube-api-access-tdf7l") pod "cfc51ead-a2f4-465a-aa9d-7c3f075c06e2" (UID: "cfc51ead-a2f4-465a-aa9d-7c3f075c06e2"). InnerVolumeSpecName "kube-api-access-tdf7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 07:06:44.293787 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.293751 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls"] Jul 30 07:06:44.293989 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.293965 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="registry" containerID="cri-o://87cabcc106ae4ece0217010f62bd021afcbf83cc581c043eee9be0bd0b2092d6" gracePeriod=30 Jul 30 07:06:44.294042 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.294011 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="online" containerID="cri-o://0397db14c39d2aa0c77b1799187192bccedcfffcba4e1207084fcf2d87fe5eec" gracePeriod=30 Jul 30 07:06:44.300396 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.300369 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r"] Jul 30 07:06:44.300505 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.300487 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdf7l\" (UniqueName: \"kubernetes.io/projected/cfc51ead-a2f4-465a-aa9d-7c3f075c06e2-kube-api-access-tdf7l\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:06:44.300621 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.300594 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="registry" containerID="cri-o://87df565c46543acdfdba9bec0357e4615d186b61b0bf8a3c5a52961e5d718bfa" gracePeriod=30 Jul 30 07:06:44.300778 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.300622 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="online" containerID="cri-o://23df4edaa55a9b450a123b27388643f92d21b7fd7f9f7b72991670cf69b4ef2d" gracePeriod=30 Jul 30 07:06:44.647290 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.647251 2575 generic.go:356] "Generic (PLEG): container finished" podID="cfc51ead-a2f4-465a-aa9d-7c3f075c06e2" containerID="2ccfe685ecde79f9dde6e87c821a9e96613b40d1374672df9271a5358fc7361d" exitCode=0 Jul 30 07:06:44.647290 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.647296 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4" Jul 30 07:06:44.647552 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.647331 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4" event={"ID":"cfc51ead-a2f4-465a-aa9d-7c3f075c06e2","Type":"ContainerDied","Data":"2ccfe685ecde79f9dde6e87c821a9e96613b40d1374672df9271a5358fc7361d"} Jul 30 07:06:44.647552 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.647393 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4" event={"ID":"cfc51ead-a2f4-465a-aa9d-7c3f075c06e2","Type":"ContainerDied","Data":"9c806b96ef8d9631b23ea1bd3d715ebf3167422cd9119dd74725d2cd0b03a89b"} Jul 30 07:06:44.647552 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.647416 2575 scope.go:117] "RemoveContainer" containerID="2ccfe685ecde79f9dde6e87c821a9e96613b40d1374672df9271a5358fc7361d" Jul 30 07:06:44.657557 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.657534 2575 scope.go:117] "RemoveContainer" containerID="2ccfe685ecde79f9dde6e87c821a9e96613b40d1374672df9271a5358fc7361d" Jul 30 07:06:44.657888 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:06:44.657867 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ccfe685ecde79f9dde6e87c821a9e96613b40d1374672df9271a5358fc7361d\": container with ID starting with 2ccfe685ecde79f9dde6e87c821a9e96613b40d1374672df9271a5358fc7361d not found: ID does not exist" containerID="2ccfe685ecde79f9dde6e87c821a9e96613b40d1374672df9271a5358fc7361d" Jul 30 07:06:44.657989 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.657895 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ccfe685ecde79f9dde6e87c821a9e96613b40d1374672df9271a5358fc7361d"} err="failed to get container status \"2ccfe685ecde79f9dde6e87c821a9e96613b40d1374672df9271a5358fc7361d\": rpc error: code = NotFound desc = could not find container \"2ccfe685ecde79f9dde6e87c821a9e96613b40d1374672df9271a5358fc7361d\": container with ID starting with 2ccfe685ecde79f9dde6e87c821a9e96613b40d1374672df9271a5358fc7361d not found: ID does not exist" Jul 30 07:06:44.691903 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.691859 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4"] Jul 30 07:06:44.696381 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:44.696347 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7884ccc455-vxvq4"] Jul 30 07:06:45.175632 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:45.175604 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfc51ead-a2f4-465a-aa9d-7c3f075c06e2" path="/var/lib/kubelet/pods/cfc51ead-a2f4-465a-aa9d-7c3f075c06e2/volumes" Jul 30 07:06:45.653100 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:45.653072 2575 generic.go:356] "Generic (PLEG): container finished" podID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerID="23df4edaa55a9b450a123b27388643f92d21b7fd7f9f7b72991670cf69b4ef2d" exitCode=0 Jul 30 07:06:45.653287 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:45.653151 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" event={"ID":"0bbf6246-9fe0-4d1d-9899-de59549688fd","Type":"ContainerDied","Data":"23df4edaa55a9b450a123b27388643f92d21b7fd7f9f7b72991670cf69b4ef2d"} Jul 30 07:06:45.656038 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:45.656016 2575 generic.go:356] "Generic (PLEG): container finished" podID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerID="0397db14c39d2aa0c77b1799187192bccedcfffcba4e1207084fcf2d87fe5eec" exitCode=0 Jul 30 07:06:45.656156 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:45.656044 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" event={"ID":"36be7fcf-d199-4dd1-9083-2a708b5765e7","Type":"ContainerDied","Data":"0397db14c39d2aa0c77b1799187192bccedcfffcba4e1207084fcf2d87fe5eec"} Jul 30 07:06:46.310242 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.310197 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Jul 30 07:06:46.379646 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.379613 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-s5vz2"] Jul 30 07:06:46.379849 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.379826 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-s5vz2" podUID="feca6faf-a9fc-489f-80fe-e04807eb4f6d" containerName="postgres" containerID="cri-o://17ef5388e3337b97b4bf57ecdc5ba261b898f416ea1f8bce3a0e264047ffdbcc" gracePeriod=30 Jul 30 07:06:46.388244 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.386414 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-8bmr9"] Jul 30 07:06:46.388244 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.386590 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-8bmr9" podUID="87273ba1-8337-4b11-82e4-171df0c13e17" containerName="redis" containerID="cri-o://ff17c24a6f3d887910cde52f4903a6be7b49284ead127a7a0a8a39cf9f31a03e" gracePeriod=30 Jul 30 07:06:46.655241 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.655217 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-s5vz2" Jul 30 07:06:46.657880 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.657861 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-8bmr9" Jul 30 07:06:46.661727 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.661704 2575 generic.go:356] "Generic (PLEG): container finished" podID="87273ba1-8337-4b11-82e4-171df0c13e17" containerID="ff17c24a6f3d887910cde52f4903a6be7b49284ead127a7a0a8a39cf9f31a03e" exitCode=0 Jul 30 07:06:46.661814 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.661735 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-8bmr9" Jul 30 07:06:46.661814 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.661767 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-8bmr9" event={"ID":"87273ba1-8337-4b11-82e4-171df0c13e17","Type":"ContainerDied","Data":"ff17c24a6f3d887910cde52f4903a6be7b49284ead127a7a0a8a39cf9f31a03e"} Jul 30 07:06:46.661814 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.661803 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-8bmr9" event={"ID":"87273ba1-8337-4b11-82e4-171df0c13e17","Type":"ContainerDied","Data":"cd551d43cbd079bb13dbbe88ea621d4c3ec8293796733d9d5645f04c781401ee"} Jul 30 07:06:46.661945 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.661825 2575 scope.go:117] "RemoveContainer" containerID="ff17c24a6f3d887910cde52f4903a6be7b49284ead127a7a0a8a39cf9f31a03e" Jul 30 07:06:46.665699 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.663493 2575 generic.go:356] "Generic (PLEG): container finished" podID="feca6faf-a9fc-489f-80fe-e04807eb4f6d" containerID="17ef5388e3337b97b4bf57ecdc5ba261b898f416ea1f8bce3a0e264047ffdbcc" exitCode=0 Jul 30 07:06:46.665699 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.663559 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-s5vz2" event={"ID":"feca6faf-a9fc-489f-80fe-e04807eb4f6d","Type":"ContainerDied","Data":"17ef5388e3337b97b4bf57ecdc5ba261b898f416ea1f8bce3a0e264047ffdbcc"} Jul 30 07:06:46.665699 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.663591 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-s5vz2" event={"ID":"feca6faf-a9fc-489f-80fe-e04807eb4f6d","Type":"ContainerDied","Data":"ecdbbcd7b4339c1e1d6adb80b3382dfb2e58649f7887e493abfb123a05b108a1"} Jul 30 07:06:46.665699 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.663688 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-s5vz2" Jul 30 07:06:46.674253 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.674230 2575 scope.go:117] "RemoveContainer" containerID="ff17c24a6f3d887910cde52f4903a6be7b49284ead127a7a0a8a39cf9f31a03e" Jul 30 07:06:46.674881 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:06:46.674735 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff17c24a6f3d887910cde52f4903a6be7b49284ead127a7a0a8a39cf9f31a03e\": container with ID starting with ff17c24a6f3d887910cde52f4903a6be7b49284ead127a7a0a8a39cf9f31a03e not found: ID does not exist" containerID="ff17c24a6f3d887910cde52f4903a6be7b49284ead127a7a0a8a39cf9f31a03e" Jul 30 07:06:46.674881 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.674775 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff17c24a6f3d887910cde52f4903a6be7b49284ead127a7a0a8a39cf9f31a03e"} err="failed to get container status \"ff17c24a6f3d887910cde52f4903a6be7b49284ead127a7a0a8a39cf9f31a03e\": rpc error: code = NotFound desc = could not find container \"ff17c24a6f3d887910cde52f4903a6be7b49284ead127a7a0a8a39cf9f31a03e\": container with ID starting with ff17c24a6f3d887910cde52f4903a6be7b49284ead127a7a0a8a39cf9f31a03e not found: ID does not exist" Jul 30 07:06:46.674881 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.674799 2575 scope.go:117] "RemoveContainer" containerID="17ef5388e3337b97b4bf57ecdc5ba261b898f416ea1f8bce3a0e264047ffdbcc" Jul 30 07:06:46.684844 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.684824 2575 scope.go:117] "RemoveContainer" containerID="17ef5388e3337b97b4bf57ecdc5ba261b898f416ea1f8bce3a0e264047ffdbcc" Jul 30 07:06:46.685159 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:06:46.685136 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17ef5388e3337b97b4bf57ecdc5ba261b898f416ea1f8bce3a0e264047ffdbcc\": container with ID starting with 17ef5388e3337b97b4bf57ecdc5ba261b898f416ea1f8bce3a0e264047ffdbcc not found: ID does not exist" containerID="17ef5388e3337b97b4bf57ecdc5ba261b898f416ea1f8bce3a0e264047ffdbcc" Jul 30 07:06:46.685214 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.685168 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17ef5388e3337b97b4bf57ecdc5ba261b898f416ea1f8bce3a0e264047ffdbcc"} err="failed to get container status \"17ef5388e3337b97b4bf57ecdc5ba261b898f416ea1f8bce3a0e264047ffdbcc\": rpc error: code = NotFound desc = could not find container \"17ef5388e3337b97b4bf57ecdc5ba261b898f416ea1f8bce3a0e264047ffdbcc\": container with ID starting with 17ef5388e3337b97b4bf57ecdc5ba261b898f416ea1f8bce3a0e264047ffdbcc not found: ID does not exist" Jul 30 07:06:46.719976 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.719948 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/feca6faf-a9fc-489f-80fe-e04807eb4f6d-postgresdata\") pod \"feca6faf-a9fc-489f-80fe-e04807eb4f6d\" (UID: \"feca6faf-a9fc-489f-80fe-e04807eb4f6d\") " Jul 30 07:06:46.720167 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.720004 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9c4bn\" (UniqueName: \"kubernetes.io/projected/feca6faf-a9fc-489f-80fe-e04807eb4f6d-kube-api-access-9c4bn\") pod \"feca6faf-a9fc-489f-80fe-e04807eb4f6d\" (UID: \"feca6faf-a9fc-489f-80fe-e04807eb4f6d\") " Jul 30 07:06:46.720167 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.720033 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nmcb\" (UniqueName: \"kubernetes.io/projected/87273ba1-8337-4b11-82e4-171df0c13e17-kube-api-access-9nmcb\") pod \"87273ba1-8337-4b11-82e4-171df0c13e17\" (UID: \"87273ba1-8337-4b11-82e4-171df0c13e17\") " Jul 30 07:06:46.722341 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.722308 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87273ba1-8337-4b11-82e4-171df0c13e17-kube-api-access-9nmcb" (OuterVolumeSpecName: "kube-api-access-9nmcb") pod "87273ba1-8337-4b11-82e4-171df0c13e17" (UID: "87273ba1-8337-4b11-82e4-171df0c13e17"). InnerVolumeSpecName "kube-api-access-9nmcb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 07:06:46.722472 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.722418 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feca6faf-a9fc-489f-80fe-e04807eb4f6d-kube-api-access-9c4bn" (OuterVolumeSpecName: "kube-api-access-9c4bn") pod "feca6faf-a9fc-489f-80fe-e04807eb4f6d" (UID: "feca6faf-a9fc-489f-80fe-e04807eb4f6d"). InnerVolumeSpecName "kube-api-access-9c4bn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 07:06:46.751233 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.751204 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feca6faf-a9fc-489f-80fe-e04807eb4f6d-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "feca6faf-a9fc-489f-80fe-e04807eb4f6d" (UID: "feca6faf-a9fc-489f-80fe-e04807eb4f6d"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 07:06:46.820651 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.820621 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nmcb\" (UniqueName: \"kubernetes.io/projected/87273ba1-8337-4b11-82e4-171df0c13e17-kube-api-access-9nmcb\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:06:46.820651 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.820650 2575 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/feca6faf-a9fc-489f-80fe-e04807eb4f6d-postgresdata\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:06:46.820837 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.820685 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9c4bn\" (UniqueName: \"kubernetes.io/projected/feca6faf-a9fc-489f-80fe-e04807eb4f6d-kube-api-access-9c4bn\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:06:46.982910 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.982882 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-8bmr9"] Jul 30 07:06:46.986724 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.986701 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-8bmr9"] Jul 30 07:06:46.998508 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:46.998485 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-s5vz2"] Jul 30 07:06:47.001867 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:47.001849 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-s5vz2"] Jul 30 07:06:47.176203 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:47.176170 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87273ba1-8337-4b11-82e4-171df0c13e17" path="/var/lib/kubelet/pods/87273ba1-8337-4b11-82e4-171df0c13e17/volumes" Jul 30 07:06:47.176510 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:47.176497 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="feca6faf-a9fc-489f-80fe-e04807eb4f6d" path="/var/lib/kubelet/pods/feca6faf-a9fc-489f-80fe-e04807eb4f6d/volumes" Jul 30 07:06:48.010054 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:48.010014 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Jul 30 07:06:56.309913 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:56.309863 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Jul 30 07:06:58.009790 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:06:58.009742 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Jul 30 07:07:06.310181 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:06.310136 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="online" probeResult="failure" output="Get \"https://10.132.0.35:6567/health\": dial tcp 10.132.0.35:6567: connect: connection refused" Jul 30 07:07:06.310566 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:06.310262 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:07:08.009413 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:08.009376 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="online" probeResult="failure" output="Get \"https://10.132.0.34:6567/health\": dial tcp 10.132.0.34:6567: connect: connection refused" Jul 30 07:07:08.009787 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:08.009534 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:07:14.780372 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:14.780338 2575 generic.go:356] "Generic (PLEG): container finished" podID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerID="87cabcc106ae4ece0217010f62bd021afcbf83cc581c043eee9be0bd0b2092d6" exitCode=137 Jul 30 07:07:14.780797 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:14.780412 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" event={"ID":"36be7fcf-d199-4dd1-9083-2a708b5765e7","Type":"ContainerDied","Data":"87cabcc106ae4ece0217010f62bd021afcbf83cc581c043eee9be0bd0b2092d6"} Jul 30 07:07:14.782816 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:14.782791 2575 generic.go:356] "Generic (PLEG): container finished" podID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerID="87df565c46543acdfdba9bec0357e4615d186b61b0bf8a3c5a52961e5d718bfa" exitCode=137 Jul 30 07:07:14.782939 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:14.782837 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" event={"ID":"0bbf6246-9fe0-4d1d-9899-de59549688fd","Type":"ContainerDied","Data":"87df565c46543acdfdba9bec0357e4615d186b61b0bf8a3c5a52961e5d718bfa"} Jul 30 07:07:14.971329 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:14.971302 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:07:14.974294 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:14.974275 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:07:15.160336 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.160295 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jg76\" (UniqueName: \"kubernetes.io/projected/0bbf6246-9fe0-4d1d-9899-de59549688fd-kube-api-access-5jg76\") pod \"0bbf6246-9fe0-4d1d-9899-de59549688fd\" (UID: \"0bbf6246-9fe0-4d1d-9899-de59549688fd\") " Jul 30 07:07:15.160336 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.160343 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/36be7fcf-d199-4dd1-9083-2a708b5765e7-feast-data\") pod \"36be7fcf-d199-4dd1-9083-2a708b5765e7\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " Jul 30 07:07:15.160574 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.160378 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kmr9\" (UniqueName: \"kubernetes.io/projected/36be7fcf-d199-4dd1-9083-2a708b5765e7-kube-api-access-4kmr9\") pod \"36be7fcf-d199-4dd1-9083-2a708b5765e7\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " Jul 30 07:07:15.160574 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.160409 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/36be7fcf-d199-4dd1-9083-2a708b5765e7-online-tls\") pod \"36be7fcf-d199-4dd1-9083-2a708b5765e7\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " Jul 30 07:07:15.160574 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.160437 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/36be7fcf-d199-4dd1-9083-2a708b5765e7-registry-tls\") pod \"36be7fcf-d199-4dd1-9083-2a708b5765e7\" (UID: \"36be7fcf-d199-4dd1-9083-2a708b5765e7\") " Jul 30 07:07:15.160574 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.160476 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0bbf6246-9fe0-4d1d-9899-de59549688fd-feast-data\") pod \"0bbf6246-9fe0-4d1d-9899-de59549688fd\" (UID: \"0bbf6246-9fe0-4d1d-9899-de59549688fd\") " Jul 30 07:07:15.160574 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.160526 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0bbf6246-9fe0-4d1d-9899-de59549688fd-online-tls\") pod \"0bbf6246-9fe0-4d1d-9899-de59549688fd\" (UID: \"0bbf6246-9fe0-4d1d-9899-de59549688fd\") " Jul 30 07:07:15.160574 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.160550 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0bbf6246-9fe0-4d1d-9899-de59549688fd-registry-tls\") pod \"0bbf6246-9fe0-4d1d-9899-de59549688fd\" (UID: \"0bbf6246-9fe0-4d1d-9899-de59549688fd\") " Jul 30 07:07:15.161140 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.161071 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bbf6246-9fe0-4d1d-9899-de59549688fd-feast-data" (OuterVolumeSpecName: "feast-data") pod "0bbf6246-9fe0-4d1d-9899-de59549688fd" (UID: "0bbf6246-9fe0-4d1d-9899-de59549688fd"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 07:07:15.162818 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.162771 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36be7fcf-d199-4dd1-9083-2a708b5765e7-kube-api-access-4kmr9" (OuterVolumeSpecName: "kube-api-access-4kmr9") pod "36be7fcf-d199-4dd1-9083-2a708b5765e7" (UID: "36be7fcf-d199-4dd1-9083-2a708b5765e7"). InnerVolumeSpecName "kube-api-access-4kmr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 07:07:15.163208 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.163185 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36be7fcf-d199-4dd1-9083-2a708b5765e7-online-tls" (OuterVolumeSpecName: "online-tls") pod "36be7fcf-d199-4dd1-9083-2a708b5765e7" (UID: "36be7fcf-d199-4dd1-9083-2a708b5765e7"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:07:15.163290 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.163199 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bbf6246-9fe0-4d1d-9899-de59549688fd-kube-api-access-5jg76" (OuterVolumeSpecName: "kube-api-access-5jg76") pod "0bbf6246-9fe0-4d1d-9899-de59549688fd" (UID: "0bbf6246-9fe0-4d1d-9899-de59549688fd"). InnerVolumeSpecName "kube-api-access-5jg76". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 07:07:15.163333 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.163287 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bbf6246-9fe0-4d1d-9899-de59549688fd-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0bbf6246-9fe0-4d1d-9899-de59549688fd" (UID: "0bbf6246-9fe0-4d1d-9899-de59549688fd"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:07:15.163423 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.163401 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36be7fcf-d199-4dd1-9083-2a708b5765e7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "36be7fcf-d199-4dd1-9083-2a708b5765e7" (UID: "36be7fcf-d199-4dd1-9083-2a708b5765e7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:07:15.163476 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.163401 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bbf6246-9fe0-4d1d-9899-de59549688fd-online-tls" (OuterVolumeSpecName: "online-tls") pod "0bbf6246-9fe0-4d1d-9899-de59549688fd" (UID: "0bbf6246-9fe0-4d1d-9899-de59549688fd"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:07:15.170298 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.170276 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36be7fcf-d199-4dd1-9083-2a708b5765e7-feast-data" (OuterVolumeSpecName: "feast-data") pod "36be7fcf-d199-4dd1-9083-2a708b5765e7" (UID: "36be7fcf-d199-4dd1-9083-2a708b5765e7"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 07:07:15.261910 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.261876 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kmr9\" (UniqueName: \"kubernetes.io/projected/36be7fcf-d199-4dd1-9083-2a708b5765e7-kube-api-access-4kmr9\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:07:15.262058 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.261921 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/36be7fcf-d199-4dd1-9083-2a708b5765e7-online-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:07:15.262058 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.261931 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/36be7fcf-d199-4dd1-9083-2a708b5765e7-registry-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:07:15.262058 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.261939 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0bbf6246-9fe0-4d1d-9899-de59549688fd-feast-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:07:15.262058 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.261946 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0bbf6246-9fe0-4d1d-9899-de59549688fd-online-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:07:15.262058 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.261953 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0bbf6246-9fe0-4d1d-9899-de59549688fd-registry-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:07:15.262058 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.261961 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jg76\" (UniqueName: \"kubernetes.io/projected/0bbf6246-9fe0-4d1d-9899-de59549688fd-kube-api-access-5jg76\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:07:15.262058 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.261968 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/36be7fcf-d199-4dd1-9083-2a708b5765e7-feast-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:07:15.790476 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.790445 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" Jul 30 07:07:15.790947 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.790451 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r" event={"ID":"0bbf6246-9fe0-4d1d-9899-de59549688fd","Type":"ContainerDied","Data":"8cb1e209467ca16bcd0b278ceced883a228f1ebd73e100f9d9d9520cf14cbd7c"} Jul 30 07:07:15.790947 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.790601 2575 scope.go:117] "RemoveContainer" containerID="23df4edaa55a9b450a123b27388643f92d21b7fd7f9f7b72991670cf69b4ef2d" Jul 30 07:07:15.793242 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.793220 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" event={"ID":"36be7fcf-d199-4dd1-9083-2a708b5765e7","Type":"ContainerDied","Data":"c809da6ac5fe6aa3f15a0cf6d4a07a0f02addf27b5d38717e5ce37470b83a57f"} Jul 30 07:07:15.793359 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.793281 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls" Jul 30 07:07:15.800487 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.800469 2575 scope.go:117] "RemoveContainer" containerID="87df565c46543acdfdba9bec0357e4615d186b61b0bf8a3c5a52961e5d718bfa" Jul 30 07:07:15.808767 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.808748 2575 scope.go:117] "RemoveContainer" containerID="87a8ea09f3bd37d29af9c46235eb9e28207514f889178e7dbf83d12b7cd1c85b" Jul 30 07:07:15.811527 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.811502 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls"] Jul 30 07:07:15.816929 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.816914 2575 scope.go:117] "RemoveContainer" containerID="31bb4830ca18b13303c5c4cc0013f80573d2c02e4a4dec519a82e7438abf090f" Jul 30 07:07:15.817544 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.817525 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-f7b5fcf5c-7xsls"] Jul 30 07:07:15.824383 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.824368 2575 scope.go:117] "RemoveContainer" containerID="0397db14c39d2aa0c77b1799187192bccedcfffcba4e1207084fcf2d87fe5eec" Jul 30 07:07:15.829984 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.829963 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r"] Jul 30 07:07:15.832285 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.832266 2575 scope.go:117] "RemoveContainer" containerID="87cabcc106ae4ece0217010f62bd021afcbf83cc581c043eee9be0bd0b2092d6" Jul 30 07:07:15.833763 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.833742 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7b5457bb66-r6d7r"] Jul 30 07:07:15.841572 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.841550 2575 scope.go:117] "RemoveContainer" containerID="607308c17e63697c843c6c0134b3d2b06da12f65108d14b4c448980d147a7df1" Jul 30 07:07:15.849326 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:15.849305 2575 scope.go:117] "RemoveContainer" containerID="524e4685755a3299cc3a035074085ee9bee4190b3c40b2af9436a58ebe453d9c" Jul 30 07:07:17.175961 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:17.175928 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" path="/var/lib/kubelet/pods/0bbf6246-9fe0-4d1d-9899-de59549688fd/volumes" Jul 30 07:07:17.176411 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:17.176398 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" path="/var/lib/kubelet/pods/36be7fcf-d199-4dd1-9083-2a708b5765e7/volumes" Jul 30 07:07:19.568386 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568352 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg"] Jul 30 07:07:19.568777 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568691 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="feast-init" Jul 30 07:07:19.568777 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568703 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="feast-init" Jul 30 07:07:19.568777 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568712 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="feast-apply" Jul 30 07:07:19.568777 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568719 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="feast-apply" Jul 30 07:07:19.568777 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568732 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feca6faf-a9fc-489f-80fe-e04807eb4f6d" containerName="postgres" Jul 30 07:07:19.568777 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568739 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="feca6faf-a9fc-489f-80fe-e04807eb4f6d" containerName="postgres" Jul 30 07:07:19.568777 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568745 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="registry" Jul 30 07:07:19.568777 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568750 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="registry" Jul 30 07:07:19.568777 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568757 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="online" Jul 30 07:07:19.568777 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568762 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="online" Jul 30 07:07:19.568777 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568769 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="feast-init" Jul 30 07:07:19.568777 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568773 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="feast-init" Jul 30 07:07:19.568777 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568780 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87273ba1-8337-4b11-82e4-171df0c13e17" containerName="redis" Jul 30 07:07:19.568777 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568785 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="87273ba1-8337-4b11-82e4-171df0c13e17" containerName="redis" Jul 30 07:07:19.569195 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568795 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfc51ead-a2f4-465a-aa9d-7c3f075c06e2" containerName="manager" Jul 30 07:07:19.569195 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568799 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfc51ead-a2f4-465a-aa9d-7c3f075c06e2" containerName="manager" Jul 30 07:07:19.569195 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568805 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="feast-apply" Jul 30 07:07:19.569195 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568810 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="feast-apply" Jul 30 07:07:19.569195 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568815 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="registry" Jul 30 07:07:19.569195 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568820 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="registry" Jul 30 07:07:19.569195 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568825 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="online" Jul 30 07:07:19.569195 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568830 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="online" Jul 30 07:07:19.569195 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568884 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="registry" Jul 30 07:07:19.569195 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568891 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bbf6246-9fe0-4d1d-9899-de59549688fd" containerName="online" Jul 30 07:07:19.569195 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568897 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="online" Jul 30 07:07:19.569195 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568902 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfc51ead-a2f4-465a-aa9d-7c3f075c06e2" containerName="manager" Jul 30 07:07:19.569195 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568907 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="feca6faf-a9fc-489f-80fe-e04807eb4f6d" containerName="postgres" Jul 30 07:07:19.569195 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568913 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="36be7fcf-d199-4dd1-9083-2a708b5765e7" containerName="registry" Jul 30 07:07:19.569195 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.568918 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="87273ba1-8337-4b11-82e4-171df0c13e17" containerName="redis" Jul 30 07:07:19.574600 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.574582 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:19.577414 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.577393 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jul 30 07:07:19.577542 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.577445 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 30 07:07:19.577712 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.577698 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-8jm6c\"" Jul 30 07:07:19.577712 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.577704 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jul 30 07:07:19.577955 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.577938 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jul 30 07:07:19.577955 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.577947 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 30 07:07:19.578104 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.577949 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jul 30 07:07:19.585902 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.585878 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg"] Jul 30 07:07:19.594936 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.594915 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:19.595047 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.594950 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:19.595047 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.594975 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f76s\" (UniqueName: \"kubernetes.io/projected/c203cbd4-2d83-4496-aea4-669d5307872d-kube-api-access-6f76s\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:19.595125 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.595050 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:19.595125 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.595090 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:19.595125 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.595108 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c203cbd4-2d83-4496-aea4-669d5307872d-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:19.695739 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.695703 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:19.695739 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.695749 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:19.695985 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.695767 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c203cbd4-2d83-4496-aea4-669d5307872d-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:19.695985 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.695800 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:19.695985 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.695832 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:19.695985 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.695858 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f76s\" (UniqueName: \"kubernetes.io/projected/c203cbd4-2d83-4496-aea4-669d5307872d-kube-api-access-6f76s\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:19.695985 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:07:19.695870 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Jul 30 07:07:19.695985 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:07:19.695930 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Jul 30 07:07:19.695985 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:07:19.695871 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Jul 30 07:07:19.695985 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:07:19.695950 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Jul 30 07:07:19.695985 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:07:19.695964 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-ui-tls podName:c203cbd4-2d83-4496-aea4-669d5307872d nodeName:}" failed. No retries permitted until 2026-07-30 07:07:20.195942407 +0000 UTC m=+1401.552946555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-ui-tls") pod "feast-simple-feast-setup-777d6cc998-5hgpg" (UID: "c203cbd4-2d83-4496-aea4-669d5307872d") : secret "feast-simple-feast-setup-ui-tls" not found Jul 30 07:07:19.696312 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:07:19.695996 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-online-tls podName:c203cbd4-2d83-4496-aea4-669d5307872d nodeName:}" failed. No retries permitted until 2026-07-30 07:07:20.19597892 +0000 UTC m=+1401.552983061 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-online-tls") pod "feast-simple-feast-setup-777d6cc998-5hgpg" (UID: "c203cbd4-2d83-4496-aea4-669d5307872d") : secret "feast-simple-feast-setup-online-tls" not found Jul 30 07:07:19.696312 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:07:19.696019 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-registry-tls podName:c203cbd4-2d83-4496-aea4-669d5307872d nodeName:}" failed. No retries permitted until 2026-07-30 07:07:20.196003836 +0000 UTC m=+1401.553007999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-registry-tls") pod "feast-simple-feast-setup-777d6cc998-5hgpg" (UID: "c203cbd4-2d83-4496-aea4-669d5307872d") : secret "feast-simple-feast-setup-registry-tls" not found Jul 30 07:07:19.696312 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:07:19.696044 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-offline-tls podName:c203cbd4-2d83-4496-aea4-669d5307872d nodeName:}" failed. No retries permitted until 2026-07-30 07:07:20.196035099 +0000 UTC m=+1401.553039251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-offline-tls") pod "feast-simple-feast-setup-777d6cc998-5hgpg" (UID: "c203cbd4-2d83-4496-aea4-669d5307872d") : secret "feast-simple-feast-setup-offline-tls" not found Jul 30 07:07:19.696312 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.696136 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c203cbd4-2d83-4496-aea4-669d5307872d-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:19.707255 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:19.707233 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f76s\" (UniqueName: \"kubernetes.io/projected/c203cbd4-2d83-4496-aea4-669d5307872d-kube-api-access-6f76s\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:20.199820 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:20.199783 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:20.200039 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:20.199882 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:20.200039 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:20.199968 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:20.200039 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:20.200028 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:20.202483 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:20.202458 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:20.203070 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:20.203045 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:20.203160 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:20.203065 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:20.203160 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:20.203112 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-5hgpg\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:20.499544 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:20.499441 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:20.639326 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:20.639299 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg"] Jul 30 07:07:20.640742 ip-10-0-143-185 kubenswrapper[2575]: W0730 07:07:20.640719 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc203cbd4_2d83_4496_aea4_669d5307872d.slice/crio-da79f211077b0b1e2af1167518fdd3d30e3beb6faea739898d6fd9348decf802 WatchSource:0}: Error finding container da79f211077b0b1e2af1167518fdd3d30e3beb6faea739898d6fd9348decf802: Status 404 returned error can't find the container with id da79f211077b0b1e2af1167518fdd3d30e3beb6faea739898d6fd9348decf802 Jul 30 07:07:20.642545 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:20.642528 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 07:07:20.814220 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:20.814125 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" event={"ID":"c203cbd4-2d83-4496-aea4-669d5307872d","Type":"ContainerStarted","Data":"da79f211077b0b1e2af1167518fdd3d30e3beb6faea739898d6fd9348decf802"} Jul 30 07:07:35.887797 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:35.887763 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" event={"ID":"c203cbd4-2d83-4496-aea4-669d5307872d","Type":"ContainerStarted","Data":"a0ea4a4d56348aa043a9854805ce831fcfc48086d8011d63f49827d6003cb50f"} Jul 30 07:07:40.909772 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:40.909741 2575 generic.go:356] "Generic (PLEG): container finished" podID="c203cbd4-2d83-4496-aea4-669d5307872d" containerID="a0ea4a4d56348aa043a9854805ce831fcfc48086d8011d63f49827d6003cb50f" exitCode=0 Jul 30 07:07:40.910232 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:40.909820 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" event={"ID":"c203cbd4-2d83-4496-aea4-669d5307872d","Type":"ContainerDied","Data":"a0ea4a4d56348aa043a9854805ce831fcfc48086d8011d63f49827d6003cb50f"} Jul 30 07:07:41.915343 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:41.915304 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" event={"ID":"c203cbd4-2d83-4496-aea4-669d5307872d","Type":"ContainerStarted","Data":"189956fcb5959d942d1c70c50b3c8d0810fbaaf6609b17971051f1e2e6565c2d"} Jul 30 07:07:46.937626 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:46.937590 2575 generic.go:356] "Generic (PLEG): container finished" podID="c203cbd4-2d83-4496-aea4-669d5307872d" containerID="189956fcb5959d942d1c70c50b3c8d0810fbaaf6609b17971051f1e2e6565c2d" exitCode=0 Jul 30 07:07:46.938152 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:46.937636 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" event={"ID":"c203cbd4-2d83-4496-aea4-669d5307872d","Type":"ContainerDied","Data":"189956fcb5959d942d1c70c50b3c8d0810fbaaf6609b17971051f1e2e6565c2d"} Jul 30 07:07:47.948069 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:47.948023 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" event={"ID":"c203cbd4-2d83-4496-aea4-669d5307872d","Type":"ContainerStarted","Data":"8fc7d17046ae637cc3f5cb30951786d9e8f4755eb0bfdbd21866b0fb8c6e6ab6"} Jul 30 07:07:47.948536 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:47.948091 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" event={"ID":"c203cbd4-2d83-4496-aea4-669d5307872d","Type":"ContainerStarted","Data":"3adfc650020ef2885352dc22e23290a0b72ca7b0a3056fe965173f21d8f5612e"} Jul 30 07:07:47.948536 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:47.948111 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" event={"ID":"c203cbd4-2d83-4496-aea4-669d5307872d","Type":"ContainerStarted","Data":"86187a32eeebec6622e21b7485897155427fdddb6e4849c342233c7c787dd846"} Jul 30 07:07:47.948536 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:47.948130 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" event={"ID":"c203cbd4-2d83-4496-aea4-669d5307872d","Type":"ContainerStarted","Data":"1c8e01ae5177d08951f6c3bfce1ef6984f588d799f195b325a5b7ab91d142712"} Jul 30 07:07:48.002082 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:48.002008 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" podStartSLOduration=14.295463984 podStartE2EDuration="29.001986771s" totalImagesPullingTime="14.706522787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 07:07:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 07:07:48.000166281 +0000 UTC m=+1429.357170481" watchObservedRunningTime="2026-07-30 07:07:48.001986771 +0000 UTC m=+1429.358990934" Jul 30 07:07:50.500742 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:50.500499 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:50.500742 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:50.500562 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:50.500742 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:50.500583 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:50.500742 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:50.500603 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:50.500742 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:50.500622 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:50.500742 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:50.500639 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:50.500742 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:50.500677 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:50.500742 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:50.500695 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:50.502566 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:50.502523 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.36:8443: connect: connection refused" Jul 30 07:07:50.502566 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:50.502551 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.36:8816: connect: connection refused" Jul 30 07:07:50.502806 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:50.502670 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.36:6567/health\": dial tcp 10.132.0.36:6567: connect: connection refused" Jul 30 07:07:50.502933 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:50.502899 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.36:6571: connect: connection refused" Jul 30 07:07:53.500589 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:53.500544 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.36:6571: connect: connection refused" Jul 30 07:07:53.501147 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:53.501126 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:53.501275 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:53.501262 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:53.501879 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:53.501842 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:53.502266 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:53.502246 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:53.506723 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:53.506707 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:53.983092 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:53.983061 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:56.500924 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:56.500896 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:07:56.992545 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:07:56.992515 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:08:59.147170 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.147134 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg"] Jul 30 07:08:59.147625 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.147437 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="registry" containerID="cri-o://1c8e01ae5177d08951f6c3bfce1ef6984f588d799f195b325a5b7ab91d142712" gracePeriod=30 Jul 30 07:08:59.147625 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.147490 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="ui" containerID="cri-o://8fc7d17046ae637cc3f5cb30951786d9e8f4755eb0bfdbd21866b0fb8c6e6ab6" gracePeriod=30 Jul 30 07:08:59.147786 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.147518 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="offline" containerID="cri-o://3adfc650020ef2885352dc22e23290a0b72ca7b0a3056fe965173f21d8f5612e" gracePeriod=30 Jul 30 07:08:59.147786 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.147535 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="online" containerID="cri-o://86187a32eeebec6622e21b7485897155427fdddb6e4849c342233c7c787dd846" gracePeriod=30 Jul 30 07:08:59.321837 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.321732 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/1.log" Jul 30 07:08:59.326107 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.326083 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/1.log" Jul 30 07:08:59.332121 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.332098 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 07:08:59.335070 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.335048 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 07:08:59.458464 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.458386 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8"] Jul 30 07:08:59.460987 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.460970 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.463636 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.463614 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-p6cck\"" Jul 30 07:08:59.476398 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.476372 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8"] Jul 30 07:08:59.642593 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.642559 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.642782 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.642618 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.642782 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.642683 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.642782 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.642766 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1038b726-2030-494d-bb7c-b982a87ea8fa-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.642897 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.642812 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.642897 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.642848 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl8s2\" (UniqueName: \"kubernetes.io/projected/1038b726-2030-494d-bb7c-b982a87ea8fa-kube-api-access-zl8s2\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.743963 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.743870 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.743963 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.743915 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.743963 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.743955 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1038b726-2030-494d-bb7c-b982a87ea8fa-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.744238 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.744061 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.744238 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.744121 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl8s2\" (UniqueName: \"kubernetes.io/projected/1038b726-2030-494d-bb7c-b982a87ea8fa-kube-api-access-zl8s2\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.744238 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.744156 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.744403 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.744356 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1038b726-2030-494d-bb7c-b982a87ea8fa-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.746863 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.746836 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.747010 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.746886 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.747104 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.747085 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.747167 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.747145 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.752844 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.752825 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl8s2\" (UniqueName: \"kubernetes.io/projected/1038b726-2030-494d-bb7c-b982a87ea8fa-kube-api-access-zl8s2\") pod \"feast-simple-feast-setup-777d6cc998-cnxg8\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.772634 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.772603 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:08:59.914125 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:08:59.914070 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8"] Jul 30 07:08:59.916712 ip-10-0-143-185 kubenswrapper[2575]: W0730 07:08:59.916682 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1038b726_2030_494d_bb7c_b982a87ea8fa.slice/crio-5d65416d94cd952759bf3d2cfefce41a4de0eca207aed41b1210930cef54dbd0 WatchSource:0}: Error finding container 5d65416d94cd952759bf3d2cfefce41a4de0eca207aed41b1210930cef54dbd0: Status 404 returned error can't find the container with id 5d65416d94cd952759bf3d2cfefce41a4de0eca207aed41b1210930cef54dbd0 Jul 30 07:09:00.244692 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:00.244637 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" event={"ID":"1038b726-2030-494d-bb7c-b982a87ea8fa","Type":"ContainerStarted","Data":"04d97408bd3783c552e388566a6e942dcace8c113bde4b71621667cfa0030129"} Jul 30 07:09:00.245154 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:00.244713 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" event={"ID":"1038b726-2030-494d-bb7c-b982a87ea8fa","Type":"ContainerStarted","Data":"5d65416d94cd952759bf3d2cfefce41a4de0eca207aed41b1210930cef54dbd0"} Jul 30 07:09:00.248889 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:00.248862 2575 generic.go:356] "Generic (PLEG): container finished" podID="c203cbd4-2d83-4496-aea4-669d5307872d" containerID="8fc7d17046ae637cc3f5cb30951786d9e8f4755eb0bfdbd21866b0fb8c6e6ab6" exitCode=0 Jul 30 07:09:00.248977 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:00.248890 2575 generic.go:356] "Generic (PLEG): container finished" podID="c203cbd4-2d83-4496-aea4-669d5307872d" containerID="86187a32eeebec6622e21b7485897155427fdddb6e4849c342233c7c787dd846" exitCode=0 Jul 30 07:09:00.248977 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:00.248918 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" event={"ID":"c203cbd4-2d83-4496-aea4-669d5307872d","Type":"ContainerDied","Data":"8fc7d17046ae637cc3f5cb30951786d9e8f4755eb0bfdbd21866b0fb8c6e6ab6"} Jul 30 07:09:00.248977 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:00.248950 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" event={"ID":"c203cbd4-2d83-4496-aea4-669d5307872d","Type":"ContainerDied","Data":"86187a32eeebec6622e21b7485897155427fdddb6e4849c342233c7c787dd846"} Jul 30 07:09:03.501972 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:03.501918 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.36:8443: connect: connection refused" Jul 30 07:09:03.980802 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:03.980750 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.36:6567/health\": dial tcp 10.132.0.36:6567: connect: connection refused" Jul 30 07:09:05.271719 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:05.271687 2575 generic.go:356] "Generic (PLEG): container finished" podID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerID="04d97408bd3783c552e388566a6e942dcace8c113bde4b71621667cfa0030129" exitCode=0 Jul 30 07:09:05.272094 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:05.271759 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" event={"ID":"1038b726-2030-494d-bb7c-b982a87ea8fa","Type":"ContainerDied","Data":"04d97408bd3783c552e388566a6e942dcace8c113bde4b71621667cfa0030129"} Jul 30 07:09:06.280721 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:06.280686 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" event={"ID":"1038b726-2030-494d-bb7c-b982a87ea8fa","Type":"ContainerStarted","Data":"b4b4c438b396cb091b8b790c1fd677183153da624284e45bcea7700c7424b11d"} Jul 30 07:09:11.304533 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:11.304437 2575 generic.go:356] "Generic (PLEG): container finished" podID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerID="b4b4c438b396cb091b8b790c1fd677183153da624284e45bcea7700c7424b11d" exitCode=0 Jul 30 07:09:11.304897 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:11.304521 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" event={"ID":"1038b726-2030-494d-bb7c-b982a87ea8fa","Type":"ContainerDied","Data":"b4b4c438b396cb091b8b790c1fd677183153da624284e45bcea7700c7424b11d"} Jul 30 07:09:12.314709 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:12.314648 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" event={"ID":"1038b726-2030-494d-bb7c-b982a87ea8fa","Type":"ContainerStarted","Data":"8071411fe6e86a56ea22f5cec73542896a9c7fb9117af2f081145a84d64ba003"} Jul 30 07:09:12.315168 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:12.314737 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" event={"ID":"1038b726-2030-494d-bb7c-b982a87ea8fa","Type":"ContainerStarted","Data":"e7e62dc65c4b11c4477e6e2632d1e5b60a1693d73bc4764ff9a5cfee173d808b"} Jul 30 07:09:12.315168 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:12.314761 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" event={"ID":"1038b726-2030-494d-bb7c-b982a87ea8fa","Type":"ContainerStarted","Data":"fa06f0b707eba1582b019374dd58bc6104a7e3907c727d691b1fa08610adc29a"} Jul 30 07:09:12.315168 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:12.314781 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" event={"ID":"1038b726-2030-494d-bb7c-b982a87ea8fa","Type":"ContainerStarted","Data":"8cc5ee3adfb70daae965df4810855cae106a179bd36eac7b4ce05a9fcc750a3d"} Jul 30 07:09:12.349073 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:12.349006 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" podStartSLOduration=13.348984385 podStartE2EDuration="13.348984385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 07:08:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 07:09:12.347194339 +0000 UTC m=+1513.704198503" watchObservedRunningTime="2026-07-30 07:09:12.348984385 +0000 UTC m=+1513.705988549" Jul 30 07:09:13.502044 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:13.501987 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.36:8443: connect: connection refused" Jul 30 07:09:13.980570 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:13.980515 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.36:6567/health\": dial tcp 10.132.0.36:6567: connect: connection refused" Jul 30 07:09:14.773145 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:14.773102 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:09:14.773621 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:14.773180 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:09:14.773621 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:14.773202 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:09:14.773621 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:14.773221 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:09:14.775443 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:14.775399 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.37:8443: connect: connection refused" Jul 30 07:09:14.775443 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:14.775425 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.37:6571: connect: connection refused" Jul 30 07:09:14.775649 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:14.775423 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.37:8816: connect: connection refused" Jul 30 07:09:14.775649 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:14.775486 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="online" probeResult="failure" output="Get \"https://10.132.0.37:6567/health\": dial tcp 10.132.0.37:6567: connect: connection refused" Jul 30 07:09:17.773506 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:17.773452 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="online" probeResult="failure" output="Get \"https://10.132.0.37:6567/health\": dial tcp 10.132.0.37:6567: connect: connection refused" Jul 30 07:09:17.774061 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:17.774032 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:09:17.774670 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:17.774624 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:09:17.774881 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:17.774866 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:09:17.774932 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:17.774897 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:09:17.774968 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:17.774954 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:09:18.346509 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:18.346476 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:09:18.346732 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:18.346531 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:09:18.346800 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:18.346751 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:09:18.347160 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:18.347138 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:09:19.773170 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:19.773121 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:09:20.778805 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:20.778772 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:09:21.362016 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:21.361980 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:09:23.502430 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:23.502394 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.36:8443: connect: connection refused" Jul 30 07:09:23.502875 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:23.502499 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:09:23.980832 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:23.980791 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="online" probeResult="failure" output="Get \"https://10.132.0.36:6567/health\": dial tcp 10.132.0.36:6567: connect: connection refused" Jul 30 07:09:23.981014 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:23.980934 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:09:29.393378 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.393350 2575 generic.go:356] "Generic (PLEG): container finished" podID="c203cbd4-2d83-4496-aea4-669d5307872d" containerID="3adfc650020ef2885352dc22e23290a0b72ca7b0a3056fe965173f21d8f5612e" exitCode=137 Jul 30 07:09:29.393378 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.393373 2575 generic.go:356] "Generic (PLEG): container finished" podID="c203cbd4-2d83-4496-aea4-669d5307872d" containerID="1c8e01ae5177d08951f6c3bfce1ef6984f588d799f195b325a5b7ab91d142712" exitCode=137 Jul 30 07:09:29.393805 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.393422 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" event={"ID":"c203cbd4-2d83-4496-aea4-669d5307872d","Type":"ContainerDied","Data":"3adfc650020ef2885352dc22e23290a0b72ca7b0a3056fe965173f21d8f5612e"} Jul 30 07:09:29.393805 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.393464 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" event={"ID":"c203cbd4-2d83-4496-aea4-669d5307872d","Type":"ContainerDied","Data":"1c8e01ae5177d08951f6c3bfce1ef6984f588d799f195b325a5b7ab91d142712"} Jul 30 07:09:29.804131 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.804108 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:09:29.913990 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.913900 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-registry-tls\") pod \"c203cbd4-2d83-4496-aea4-669d5307872d\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " Jul 30 07:09:29.913990 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.913940 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-ui-tls\") pod \"c203cbd4-2d83-4496-aea4-669d5307872d\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " Jul 30 07:09:29.913990 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.913973 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-online-tls\") pod \"c203cbd4-2d83-4496-aea4-669d5307872d\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " Jul 30 07:09:29.914278 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.914150 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-offline-tls\") pod \"c203cbd4-2d83-4496-aea4-669d5307872d\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " Jul 30 07:09:29.914278 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.914194 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6f76s\" (UniqueName: \"kubernetes.io/projected/c203cbd4-2d83-4496-aea4-669d5307872d-kube-api-access-6f76s\") pod \"c203cbd4-2d83-4496-aea4-669d5307872d\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " Jul 30 07:09:29.914278 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.914245 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c203cbd4-2d83-4496-aea4-669d5307872d-feast-data\") pod \"c203cbd4-2d83-4496-aea4-669d5307872d\" (UID: \"c203cbd4-2d83-4496-aea4-669d5307872d\") " Jul 30 07:09:29.914882 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.914854 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c203cbd4-2d83-4496-aea4-669d5307872d-feast-data" (OuterVolumeSpecName: "feast-data") pod "c203cbd4-2d83-4496-aea4-669d5307872d" (UID: "c203cbd4-2d83-4496-aea4-669d5307872d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 07:09:29.916461 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.916436 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c203cbd4-2d83-4496-aea4-669d5307872d" (UID: "c203cbd4-2d83-4496-aea4-669d5307872d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:09:29.916792 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.916754 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-online-tls" (OuterVolumeSpecName: "online-tls") pod "c203cbd4-2d83-4496-aea4-669d5307872d" (UID: "c203cbd4-2d83-4496-aea4-669d5307872d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:09:29.916792 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.916775 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "c203cbd4-2d83-4496-aea4-669d5307872d" (UID: "c203cbd4-2d83-4496-aea4-669d5307872d"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:09:29.916965 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.916925 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "c203cbd4-2d83-4496-aea4-669d5307872d" (UID: "c203cbd4-2d83-4496-aea4-669d5307872d"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:09:29.916965 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:29.916945 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c203cbd4-2d83-4496-aea4-669d5307872d-kube-api-access-6f76s" (OuterVolumeSpecName: "kube-api-access-6f76s") pod "c203cbd4-2d83-4496-aea4-669d5307872d" (UID: "c203cbd4-2d83-4496-aea4-669d5307872d"). InnerVolumeSpecName "kube-api-access-6f76s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 07:09:30.015612 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:30.015580 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-offline-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:09:30.015612 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:30.015608 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6f76s\" (UniqueName: \"kubernetes.io/projected/c203cbd4-2d83-4496-aea4-669d5307872d-kube-api-access-6f76s\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:09:30.015612 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:30.015617 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c203cbd4-2d83-4496-aea4-669d5307872d-feast-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:09:30.015848 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:30.015625 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-registry-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:09:30.015848 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:30.015633 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-ui-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:09:30.015848 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:30.015640 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c203cbd4-2d83-4496-aea4-669d5307872d-online-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:09:30.400667 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:30.400615 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" event={"ID":"c203cbd4-2d83-4496-aea4-669d5307872d","Type":"ContainerDied","Data":"da79f211077b0b1e2af1167518fdd3d30e3beb6faea739898d6fd9348decf802"} Jul 30 07:09:30.400667 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:30.400650 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg" Jul 30 07:09:30.401132 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:30.400678 2575 scope.go:117] "RemoveContainer" containerID="8fc7d17046ae637cc3f5cb30951786d9e8f4755eb0bfdbd21866b0fb8c6e6ab6" Jul 30 07:09:30.409976 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:30.409956 2575 scope.go:117] "RemoveContainer" containerID="3adfc650020ef2885352dc22e23290a0b72ca7b0a3056fe965173f21d8f5612e" Jul 30 07:09:30.418016 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:30.417995 2575 scope.go:117] "RemoveContainer" containerID="86187a32eeebec6622e21b7485897155427fdddb6e4849c342233c7c787dd846" Jul 30 07:09:30.422296 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:30.422275 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg"] Jul 30 07:09:30.426329 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:30.426307 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-5hgpg"] Jul 30 07:09:30.426745 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:30.426721 2575 scope.go:117] "RemoveContainer" containerID="1c8e01ae5177d08951f6c3bfce1ef6984f588d799f195b325a5b7ab91d142712" Jul 30 07:09:30.434274 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:30.434256 2575 scope.go:117] "RemoveContainer" containerID="189956fcb5959d942d1c70c50b3c8d0810fbaaf6609b17971051f1e2e6565c2d" Jul 30 07:09:30.441301 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:30.441286 2575 scope.go:117] "RemoveContainer" containerID="a0ea4a4d56348aa043a9854805ce831fcfc48086d8011d63f49827d6003cb50f" Jul 30 07:09:31.175490 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:09:31.175457 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" path="/var/lib/kubelet/pods/c203cbd4-2d83-4496-aea4-669d5307872d/volumes" Jul 30 07:10:23.228782 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.228746 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt"] Jul 30 07:10:23.229179 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.229041 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="registry" Jul 30 07:10:23.229179 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.229051 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="registry" Jul 30 07:10:23.229179 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.229062 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="online" Jul 30 07:10:23.229179 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.229067 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="online" Jul 30 07:10:23.229179 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.229077 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="feast-apply" Jul 30 07:10:23.229179 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.229084 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="feast-apply" Jul 30 07:10:23.229179 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.229102 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="ui" Jul 30 07:10:23.229179 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.229107 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="ui" Jul 30 07:10:23.229179 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.229115 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="feast-init" Jul 30 07:10:23.229179 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.229122 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="feast-init" Jul 30 07:10:23.229179 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.229129 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="offline" Jul 30 07:10:23.229179 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.229134 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="offline" Jul 30 07:10:23.229179 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.229182 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="registry" Jul 30 07:10:23.229563 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.229190 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="ui" Jul 30 07:10:23.229563 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.229196 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="online" Jul 30 07:10:23.229563 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.229202 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c203cbd4-2d83-4496-aea4-669d5307872d" containerName="offline" Jul 30 07:10:23.233648 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.233622 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.236577 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.236553 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jul 30 07:10:23.237631 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.237613 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-6m78x\"" Jul 30 07:10:23.237746 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.237683 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jul 30 07:10:23.237805 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.237765 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jul 30 07:10:23.237805 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.237790 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jul 30 07:10:23.237916 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.237905 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jul 30 07:10:23.237977 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.237919 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jul 30 07:10:23.247978 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.247574 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt"] Jul 30 07:10:23.332372 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.332335 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.332372 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.332374 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.332588 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.332399 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.332588 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.332462 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e625d535-6890-4769-b524-915ffdee3bdd-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.332588 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.332500 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e625d535-6890-4769-b524-915ffdee3bdd-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.332588 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.332527 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6knc9\" (UniqueName: \"kubernetes.io/projected/e625d535-6890-4769-b524-915ffdee3bdd-kube-api-access-6knc9\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.433222 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.433188 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.433222 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.433228 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.433535 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.433252 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.433535 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:10:23.433351 2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found Jul 30 07:10:23.433535 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.433398 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e625d535-6890-4769-b524-915ffdee3bdd-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.433535 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:10:23.433415 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-offline-tls podName:e625d535-6890-4769-b524-915ffdee3bdd nodeName:}" failed. No retries permitted until 2026-07-30 07:10:23.933396554 +0000 UTC m=+1585.290400700 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-offline-tls") pod "feast-simple-feast-remote-setup-db6955c6c-ckddt" (UID: "e625d535-6890-4769-b524-915ffdee3bdd") : secret "feast-simple-feast-remote-setup-offline-tls" not found Jul 30 07:10:23.433535 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.433451 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e625d535-6890-4769-b524-915ffdee3bdd-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.433850 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.433586 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6knc9\" (UniqueName: \"kubernetes.io/projected/e625d535-6890-4769-b524-915ffdee3bdd-kube-api-access-6knc9\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.433924 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.433897 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e625d535-6890-4769-b524-915ffdee3bdd-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.434126 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.434109 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e625d535-6890-4769-b524-915ffdee3bdd-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.435803 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.435783 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.435908 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.435848 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.445031 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.445004 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6knc9\" (UniqueName: \"kubernetes.io/projected/e625d535-6890-4769-b524-915ffdee3bdd-kube-api-access-6knc9\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.938192 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.938155 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:23.940626 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:23.940607 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-ckddt\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:24.142932 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:24.142880 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:24.281675 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:24.281636 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt"] Jul 30 07:10:24.283018 ip-10-0-143-185 kubenswrapper[2575]: W0730 07:10:24.282991 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode625d535_6890_4769_b524_915ffdee3bdd.slice/crio-960344cb8d4392cfa745943bb1c9fb53ba8824636be23342cd8c8e1023f85106 WatchSource:0}: Error finding container 960344cb8d4392cfa745943bb1c9fb53ba8824636be23342cd8c8e1023f85106: Status 404 returned error can't find the container with id 960344cb8d4392cfa745943bb1c9fb53ba8824636be23342cd8c8e1023f85106 Jul 30 07:10:24.610788 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:24.610694 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" event={"ID":"e625d535-6890-4769-b524-915ffdee3bdd","Type":"ContainerStarted","Data":"687bfc7aecee7224871f009cb9d8aedb80be55c5ef91e879638e5c997fa7db04"} Jul 30 07:10:24.610788 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:24.610751 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" event={"ID":"e625d535-6890-4769-b524-915ffdee3bdd","Type":"ContainerStarted","Data":"960344cb8d4392cfa745943bb1c9fb53ba8824636be23342cd8c8e1023f85106"} Jul 30 07:10:30.636955 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:30.636919 2575 generic.go:356] "Generic (PLEG): container finished" podID="e625d535-6890-4769-b524-915ffdee3bdd" containerID="687bfc7aecee7224871f009cb9d8aedb80be55c5ef91e879638e5c997fa7db04" exitCode=0 Jul 30 07:10:30.637391 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:30.636999 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" event={"ID":"e625d535-6890-4769-b524-915ffdee3bdd","Type":"ContainerDied","Data":"687bfc7aecee7224871f009cb9d8aedb80be55c5ef91e879638e5c997fa7db04"} Jul 30 07:10:31.642856 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:31.642822 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" event={"ID":"e625d535-6890-4769-b524-915ffdee3bdd","Type":"ContainerStarted","Data":"fc14eb438751bb591eb77a0263eb05364327b1c230a52c7bb9f373fd05bf49ae"} Jul 30 07:10:36.664117 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:36.664082 2575 generic.go:356] "Generic (PLEG): container finished" podID="e625d535-6890-4769-b524-915ffdee3bdd" containerID="fc14eb438751bb591eb77a0263eb05364327b1c230a52c7bb9f373fd05bf49ae" exitCode=0 Jul 30 07:10:36.664626 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:36.664163 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" event={"ID":"e625d535-6890-4769-b524-915ffdee3bdd","Type":"ContainerDied","Data":"fc14eb438751bb591eb77a0263eb05364327b1c230a52c7bb9f373fd05bf49ae"} Jul 30 07:10:37.672942 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:37.672900 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" event={"ID":"e625d535-6890-4769-b524-915ffdee3bdd","Type":"ContainerStarted","Data":"ecea2cfb504d6d92738c9472685d16e252fbe5cd38b802ee1458e17b9528862d"} Jul 30 07:10:37.673411 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:37.672967 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" event={"ID":"e625d535-6890-4769-b524-915ffdee3bdd","Type":"ContainerStarted","Data":"2b844ea0d269ed4c86c7499f75d8670507239eda87495d3e600297b927aa308a"} Jul 30 07:10:37.673411 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:37.672987 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" event={"ID":"e625d535-6890-4769-b524-915ffdee3bdd","Type":"ContainerStarted","Data":"de3240f575c4dc727e093152f5d232678c7b0a161d1ea561ca420220446fcf70"} Jul 30 07:10:39.143373 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:39.143324 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:39.143897 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:39.143396 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:39.143897 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:39.143412 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:39.144982 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:39.144939 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.38:8443: connect: connection refused" Jul 30 07:10:39.145113 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:39.145000 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused" Jul 30 07:10:39.145113 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:39.144948 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.38:8816: connect: connection refused" Jul 30 07:10:42.143354 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:42.143294 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.38:8443: connect: connection refused" Jul 30 07:10:42.143880 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:42.143373 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused" Jul 30 07:10:42.143880 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:42.143294 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.38:8816: connect: connection refused" Jul 30 07:10:44.143971 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:44.143920 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:44.144372 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:44.144001 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:44.144372 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:44.144042 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:45.144531 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:45.144501 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:45.145063 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:45.144958 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:45.151103 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:45.151080 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:45.190577 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:45.190528 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" podStartSLOduration=22.190511168 podStartE2EDuration="22.190511168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 07:10:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 07:10:37.704888811 +0000 UTC m=+1599.061893010" watchObservedRunningTime="2026-07-30 07:10:45.190511168 +0000 UTC m=+1606.547515330" Jul 30 07:10:45.712409 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:45.712382 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:45.712598 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:45.712581 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:10:45.716577 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:10:45.716550 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:11:48.522729 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:48.522697 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt"] Jul 30 07:11:48.523299 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:48.522954 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="online" containerID="cri-o://de3240f575c4dc727e093152f5d232678c7b0a161d1ea561ca420220446fcf70" gracePeriod=30 Jul 30 07:11:48.523299 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:48.523055 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="ui" containerID="cri-o://ecea2cfb504d6d92738c9472685d16e252fbe5cd38b802ee1458e17b9528862d" gracePeriod=30 Jul 30 07:11:48.523299 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:48.523064 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="offline" containerID="cri-o://2b844ea0d269ed4c86c7499f75d8670507239eda87495d3e600297b927aa308a" gracePeriod=30 Jul 30 07:11:48.668847 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:48.668819 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8"] Jul 30 07:11:48.669084 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:48.669056 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="registry" containerID="cri-o://8cc5ee3adfb70daae965df4810855cae106a179bd36eac7b4ce05a9fcc750a3d" gracePeriod=30 Jul 30 07:11:48.669344 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:48.669108 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="offline" containerID="cri-o://e7e62dc65c4b11c4477e6e2632d1e5b60a1693d73bc4764ff9a5cfee173d808b" gracePeriod=30 Jul 30 07:11:48.669438 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:48.669107 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="ui" containerID="cri-o://8071411fe6e86a56ea22f5cec73542896a9c7fb9117af2f081145a84d64ba003" gracePeriod=30 Jul 30 07:11:48.669502 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:48.669144 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="online" containerID="cri-o://fa06f0b707eba1582b019374dd58bc6104a7e3907c727d691b1fa08610adc29a" gracePeriod=30 Jul 30 07:11:49.978507 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:49.978473 2575 generic.go:356] "Generic (PLEG): container finished" podID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerID="8071411fe6e86a56ea22f5cec73542896a9c7fb9117af2f081145a84d64ba003" exitCode=0 Jul 30 07:11:49.978507 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:49.978500 2575 generic.go:356] "Generic (PLEG): container finished" podID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerID="fa06f0b707eba1582b019374dd58bc6104a7e3907c727d691b1fa08610adc29a" exitCode=0 Jul 30 07:11:49.979056 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:49.978544 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" event={"ID":"1038b726-2030-494d-bb7c-b982a87ea8fa","Type":"ContainerDied","Data":"8071411fe6e86a56ea22f5cec73542896a9c7fb9117af2f081145a84d64ba003"} Jul 30 07:11:49.979056 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:49.978585 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" event={"ID":"1038b726-2030-494d-bb7c-b982a87ea8fa","Type":"ContainerDied","Data":"fa06f0b707eba1582b019374dd58bc6104a7e3907c727d691b1fa08610adc29a"} Jul 30 07:11:49.980913 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:49.980891 2575 generic.go:356] "Generic (PLEG): container finished" podID="e625d535-6890-4769-b524-915ffdee3bdd" containerID="ecea2cfb504d6d92738c9472685d16e252fbe5cd38b802ee1458e17b9528862d" exitCode=0 Jul 30 07:11:49.980913 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:49.980910 2575 generic.go:356] "Generic (PLEG): container finished" podID="e625d535-6890-4769-b524-915ffdee3bdd" containerID="de3240f575c4dc727e093152f5d232678c7b0a161d1ea561ca420220446fcf70" exitCode=0 Jul 30 07:11:49.981082 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:49.980960 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" event={"ID":"e625d535-6890-4769-b524-915ffdee3bdd","Type":"ContainerDied","Data":"ecea2cfb504d6d92738c9472685d16e252fbe5cd38b802ee1458e17b9528862d"} Jul 30 07:11:49.981082 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:49.980994 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" event={"ID":"e625d535-6890-4769-b524-915ffdee3bdd","Type":"ContainerDied","Data":"de3240f575c4dc727e093152f5d232678c7b0a161d1ea561ca420220446fcf70"} Jul 30 07:11:51.359097 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:51.359060 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="online" probeResult="failure" output="Get \"https://10.132.0.37:6567/health\": dial tcp 10.132.0.37:6567: connect: connection refused" Jul 30 07:11:55.712604 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:55.712553 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.38:8443: connect: connection refused" Jul 30 07:11:55.713013 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:55.712565 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused" Jul 30 07:11:58.347010 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:11:58.346967 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.37:8443: connect: connection refused" Jul 30 07:12:01.359750 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:01.359706 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="online" probeResult="failure" output="Get \"https://10.132.0.37:6567/health\": dial tcp 10.132.0.37:6567: connect: connection refused" Jul 30 07:12:05.712327 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:05.712271 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.38:8443: connect: connection refused" Jul 30 07:12:05.712779 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:05.712286 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused" Jul 30 07:12:08.347417 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:08.347374 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.37:8443: connect: connection refused" Jul 30 07:12:11.359255 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:11.359211 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="online" probeResult="failure" output="Get \"https://10.132.0.37:6567/health\": dial tcp 10.132.0.37:6567: connect: connection refused" Jul 30 07:12:11.359681 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:11.359344 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:12:15.712923 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:15.712876 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.38:8443: connect: connection refused" Jul 30 07:12:15.713360 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:15.712888 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="online" probeResult="failure" output="Get \"https://10.132.0.38:6567/health\": dial tcp 10.132.0.38:6567: connect: connection refused" Jul 30 07:12:15.713360 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:15.713043 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:12:15.713360 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:15.713115 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:12:18.347395 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:18.347352 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.37:8443: connect: connection refused" Jul 30 07:12:18.347899 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:18.347502 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:12:19.096887 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.096857 2575 generic.go:356] "Generic (PLEG): container finished" podID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerID="e7e62dc65c4b11c4477e6e2632d1e5b60a1693d73bc4764ff9a5cfee173d808b" exitCode=137 Jul 30 07:12:19.096887 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.096883 2575 generic.go:356] "Generic (PLEG): container finished" podID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerID="8cc5ee3adfb70daae965df4810855cae106a179bd36eac7b4ce05a9fcc750a3d" exitCode=137 Jul 30 07:12:19.097092 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.096926 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" event={"ID":"1038b726-2030-494d-bb7c-b982a87ea8fa","Type":"ContainerDied","Data":"e7e62dc65c4b11c4477e6e2632d1e5b60a1693d73bc4764ff9a5cfee173d808b"} Jul 30 07:12:19.097092 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.096970 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" event={"ID":"1038b726-2030-494d-bb7c-b982a87ea8fa","Type":"ContainerDied","Data":"8cc5ee3adfb70daae965df4810855cae106a179bd36eac7b4ce05a9fcc750a3d"} Jul 30 07:12:19.099954 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.099929 2575 generic.go:356] "Generic (PLEG): container finished" podID="e625d535-6890-4769-b524-915ffdee3bdd" containerID="2b844ea0d269ed4c86c7499f75d8670507239eda87495d3e600297b927aa308a" exitCode=137 Jul 30 07:12:19.100041 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.100004 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" event={"ID":"e625d535-6890-4769-b524-915ffdee3bdd","Type":"ContainerDied","Data":"2b844ea0d269ed4c86c7499f75d8670507239eda87495d3e600297b927aa308a"} Jul 30 07:12:19.173931 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.173901 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:12:19.194252 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.194123 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e625d535-6890-4769-b524-915ffdee3bdd-registry-tls\") pod \"e625d535-6890-4769-b524-915ffdee3bdd\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " Jul 30 07:12:19.194252 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.194153 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e625d535-6890-4769-b524-915ffdee3bdd-feast-data\") pod \"e625d535-6890-4769-b524-915ffdee3bdd\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " Jul 30 07:12:19.194252 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.194176 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-offline-tls\") pod \"e625d535-6890-4769-b524-915ffdee3bdd\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " Jul 30 07:12:19.194252 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.194201 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-online-tls\") pod \"e625d535-6890-4769-b524-915ffdee3bdd\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " Jul 30 07:12:19.194252 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.194249 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6knc9\" (UniqueName: \"kubernetes.io/projected/e625d535-6890-4769-b524-915ffdee3bdd-kube-api-access-6knc9\") pod \"e625d535-6890-4769-b524-915ffdee3bdd\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " Jul 30 07:12:19.194529 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.194312 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-ui-tls\") pod \"e625d535-6890-4769-b524-915ffdee3bdd\" (UID: \"e625d535-6890-4769-b524-915ffdee3bdd\") " Jul 30 07:12:19.194641 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.194544 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e625d535-6890-4769-b524-915ffdee3bdd-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e625d535-6890-4769-b524-915ffdee3bdd" (UID: "e625d535-6890-4769-b524-915ffdee3bdd"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 07:12:19.194641 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.194624 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/e625d535-6890-4769-b524-915ffdee3bdd-registry-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:12:19.194771 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.194733 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e625d535-6890-4769-b524-915ffdee3bdd-feast-data" (OuterVolumeSpecName: "feast-data") pod "e625d535-6890-4769-b524-915ffdee3bdd" (UID: "e625d535-6890-4769-b524-915ffdee3bdd"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 07:12:19.196826 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.196801 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e625d535-6890-4769-b524-915ffdee3bdd" (UID: "e625d535-6890-4769-b524-915ffdee3bdd"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:12:19.197279 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.197254 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-online-tls" (OuterVolumeSpecName: "online-tls") pod "e625d535-6890-4769-b524-915ffdee3bdd" (UID: "e625d535-6890-4769-b524-915ffdee3bdd"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:12:19.197279 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.197268 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e625d535-6890-4769-b524-915ffdee3bdd-kube-api-access-6knc9" (OuterVolumeSpecName: "kube-api-access-6knc9") pod "e625d535-6890-4769-b524-915ffdee3bdd" (UID: "e625d535-6890-4769-b524-915ffdee3bdd"). InnerVolumeSpecName "kube-api-access-6knc9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 07:12:19.197391 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.197268 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e625d535-6890-4769-b524-915ffdee3bdd" (UID: "e625d535-6890-4769-b524-915ffdee3bdd"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:12:19.296005 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.295924 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6knc9\" (UniqueName: \"kubernetes.io/projected/e625d535-6890-4769-b524-915ffdee3bdd-kube-api-access-6knc9\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:12:19.296005 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.295954 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-ui-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:12:19.296005 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.295968 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e625d535-6890-4769-b524-915ffdee3bdd-feast-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:12:19.296005 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.295979 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-offline-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:12:19.296005 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.295990 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e625d535-6890-4769-b524-915ffdee3bdd-online-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:12:19.329689 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.329644 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:12:19.396919 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.396889 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-ui-tls\") pod \"1038b726-2030-494d-bb7c-b982a87ea8fa\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " Jul 30 07:12:19.397411 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.396936 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-offline-tls\") pod \"1038b726-2030-494d-bb7c-b982a87ea8fa\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " Jul 30 07:12:19.397411 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.396970 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-online-tls\") pod \"1038b726-2030-494d-bb7c-b982a87ea8fa\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " Jul 30 07:12:19.397411 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.397027 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zl8s2\" (UniqueName: \"kubernetes.io/projected/1038b726-2030-494d-bb7c-b982a87ea8fa-kube-api-access-zl8s2\") pod \"1038b726-2030-494d-bb7c-b982a87ea8fa\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " Jul 30 07:12:19.397411 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.397072 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1038b726-2030-494d-bb7c-b982a87ea8fa-feast-data\") pod \"1038b726-2030-494d-bb7c-b982a87ea8fa\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " Jul 30 07:12:19.397411 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.397110 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-registry-tls\") pod \"1038b726-2030-494d-bb7c-b982a87ea8fa\" (UID: \"1038b726-2030-494d-bb7c-b982a87ea8fa\") " Jul 30 07:12:19.397878 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.397633 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1038b726-2030-494d-bb7c-b982a87ea8fa-feast-data" (OuterVolumeSpecName: "feast-data") pod "1038b726-2030-494d-bb7c-b982a87ea8fa" (UID: "1038b726-2030-494d-bb7c-b982a87ea8fa"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 07:12:19.399341 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.399297 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1038b726-2030-494d-bb7c-b982a87ea8fa-kube-api-access-zl8s2" (OuterVolumeSpecName: "kube-api-access-zl8s2") pod "1038b726-2030-494d-bb7c-b982a87ea8fa" (UID: "1038b726-2030-494d-bb7c-b982a87ea8fa"). InnerVolumeSpecName "kube-api-access-zl8s2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 07:12:19.399836 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.399803 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "1038b726-2030-494d-bb7c-b982a87ea8fa" (UID: "1038b726-2030-494d-bb7c-b982a87ea8fa"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:12:19.399836 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.399820 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-online-tls" (OuterVolumeSpecName: "online-tls") pod "1038b726-2030-494d-bb7c-b982a87ea8fa" (UID: "1038b726-2030-494d-bb7c-b982a87ea8fa"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:12:19.399953 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.399853 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "1038b726-2030-494d-bb7c-b982a87ea8fa" (UID: "1038b726-2030-494d-bb7c-b982a87ea8fa"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:12:19.399953 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.399873 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1038b726-2030-494d-bb7c-b982a87ea8fa" (UID: "1038b726-2030-494d-bb7c-b982a87ea8fa"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:12:19.498245 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.498210 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-ui-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:12:19.498245 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.498239 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-offline-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:12:19.498245 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.498249 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-online-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:12:19.498560 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.498256 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zl8s2\" (UniqueName: \"kubernetes.io/projected/1038b726-2030-494d-bb7c-b982a87ea8fa-kube-api-access-zl8s2\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:12:19.498560 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.498266 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/1038b726-2030-494d-bb7c-b982a87ea8fa-feast-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:12:19.498560 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:19.498274 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/1038b726-2030-494d-bb7c-b982a87ea8fa-registry-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:12:20.106965 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.106881 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" Jul 30 07:12:20.107133 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.106881 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8" event={"ID":"1038b726-2030-494d-bb7c-b982a87ea8fa","Type":"ContainerDied","Data":"5d65416d94cd952759bf3d2cfefce41a4de0eca207aed41b1210930cef54dbd0"} Jul 30 07:12:20.107133 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.107014 2575 scope.go:117] "RemoveContainer" containerID="8071411fe6e86a56ea22f5cec73542896a9c7fb9117af2f081145a84d64ba003" Jul 30 07:12:20.109833 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.109795 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" event={"ID":"e625d535-6890-4769-b524-915ffdee3bdd","Type":"ContainerDied","Data":"960344cb8d4392cfa745943bb1c9fb53ba8824636be23342cd8c8e1023f85106"} Jul 30 07:12:20.109833 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.109826 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt" Jul 30 07:12:20.117081 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.117066 2575 scope.go:117] "RemoveContainer" containerID="e7e62dc65c4b11c4477e6e2632d1e5b60a1693d73bc4764ff9a5cfee173d808b" Jul 30 07:12:20.124897 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.124882 2575 scope.go:117] "RemoveContainer" containerID="fa06f0b707eba1582b019374dd58bc6104a7e3907c727d691b1fa08610adc29a" Jul 30 07:12:20.132408 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.132393 2575 scope.go:117] "RemoveContainer" containerID="8cc5ee3adfb70daae965df4810855cae106a179bd36eac7b4ce05a9fcc750a3d" Jul 30 07:12:20.140004 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.139987 2575 scope.go:117] "RemoveContainer" containerID="b4b4c438b396cb091b8b790c1fd677183153da624284e45bcea7700c7424b11d" Jul 30 07:12:20.143406 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.143384 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8"] Jul 30 07:12:20.148554 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.148533 2575 scope.go:117] "RemoveContainer" containerID="04d97408bd3783c552e388566a6e942dcace8c113bde4b71621667cfa0030129" Jul 30 07:12:20.155586 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.155561 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-cnxg8"] Jul 30 07:12:20.156720 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.156699 2575 scope.go:117] "RemoveContainer" containerID="ecea2cfb504d6d92738c9472685d16e252fbe5cd38b802ee1458e17b9528862d" Jul 30 07:12:20.164727 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.164709 2575 scope.go:117] "RemoveContainer" containerID="2b844ea0d269ed4c86c7499f75d8670507239eda87495d3e600297b927aa308a" Jul 30 07:12:20.172627 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.172600 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt"] Jul 30 07:12:20.173034 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.173019 2575 scope.go:117] "RemoveContainer" containerID="de3240f575c4dc727e093152f5d232678c7b0a161d1ea561ca420220446fcf70" Jul 30 07:12:20.181051 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.181026 2575 scope.go:117] "RemoveContainer" containerID="fc14eb438751bb591eb77a0263eb05364327b1c230a52c7bb9f373fd05bf49ae" Jul 30 07:12:20.182163 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.182131 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-ckddt"] Jul 30 07:12:20.188682 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:20.188636 2575 scope.go:117] "RemoveContainer" containerID="687bfc7aecee7224871f009cb9d8aedb80be55c5ef91e879638e5c997fa7db04" Jul 30 07:12:21.175760 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:21.175725 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" path="/var/lib/kubelet/pods/1038b726-2030-494d-bb7c-b982a87ea8fa/volumes" Jul 30 07:12:21.176328 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:12:21.176315 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e625d535-6890-4769-b524-915ffdee3bdd" path="/var/lib/kubelet/pods/e625d535-6890-4769-b524-915ffdee3bdd/volumes" Jul 30 07:13:01.632520 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632489 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq"] Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632815 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="feast-init" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632825 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="feast-init" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632836 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="feast-init" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632843 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="feast-init" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632852 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="online" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632861 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="online" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632873 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="offline" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632881 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="offline" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632890 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="offline" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632895 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="offline" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632905 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="registry" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632910 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="registry" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632918 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="feast-apply" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632922 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="feast-apply" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632927 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="feast-apply" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632933 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="feast-apply" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632941 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="online" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632946 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="online" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632952 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="ui" Jul 30 07:13:01.632950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632959 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="ui" Jul 30 07:13:01.633541 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632965 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="ui" Jul 30 07:13:01.633541 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.632971 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="ui" Jul 30 07:13:01.633541 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.633031 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="registry" Jul 30 07:13:01.633541 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.633039 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="offline" Jul 30 07:13:01.633541 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.633046 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="online" Jul 30 07:13:01.633541 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.633052 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="online" Jul 30 07:13:01.633541 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.633057 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="offline" Jul 30 07:13:01.633541 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.633063 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e625d535-6890-4769-b524-915ffdee3bdd" containerName="ui" Jul 30 07:13:01.633541 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.633068 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1038b726-2030-494d-bb7c-b982a87ea8fa" containerName="ui" Jul 30 07:13:01.636309 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.636292 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.650174 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.650150 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jul 30 07:13:01.654700 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.654680 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jul 30 07:13:01.654828 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.654705 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 30 07:13:01.654828 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.654716 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-rc7q7\"" Jul 30 07:13:01.654828 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.654709 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jul 30 07:13:01.654828 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.654793 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jul 30 07:13:01.655094 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.655079 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 30 07:13:01.683835 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.683815 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq"] Jul 30 07:13:01.740993 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.740957 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.741173 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.741007 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.741173 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.741025 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.741173 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.741063 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.741173 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.741164 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqrxc\" (UniqueName: \"kubernetes.io/projected/7d6eddcb-b229-4c8c-876c-78f88f269eef-kube-api-access-mqrxc\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.741332 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.741205 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d6eddcb-b229-4c8c-876c-78f88f269eef-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.842507 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.842466 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.842507 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.842502 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.842798 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.842518 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.842798 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.842554 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqrxc\" (UniqueName: \"kubernetes.io/projected/7d6eddcb-b229-4c8c-876c-78f88f269eef-kube-api-access-mqrxc\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.842798 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.842584 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d6eddcb-b229-4c8c-876c-78f88f269eef-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.842798 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.842617 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.842798 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:13:01.842633 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Jul 30 07:13:01.842798 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:13:01.842742 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-offline-tls podName:7d6eddcb-b229-4c8c-876c-78f88f269eef nodeName:}" failed. No retries permitted until 2026-07-30 07:13:02.342710872 +0000 UTC m=+1743.699715026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-offline-tls") pod "feast-simple-feast-setup-777d6cc998-7m5hq" (UID: "7d6eddcb-b229-4c8c-876c-78f88f269eef") : secret "feast-simple-feast-setup-offline-tls" not found Jul 30 07:13:01.843106 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.843067 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d6eddcb-b229-4c8c-876c-78f88f269eef-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.845186 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.845154 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.845289 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.845246 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.845289 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.845258 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:01.852720 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:01.852700 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqrxc\" (UniqueName: \"kubernetes.io/projected/7d6eddcb-b229-4c8c-876c-78f88f269eef-kube-api-access-mqrxc\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:02.346904 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:02.346870 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:02.349342 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:02.349314 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-7m5hq\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:02.554072 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:02.554016 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:02.698311 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:02.698278 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq"] Jul 30 07:13:02.701169 ip-10-0-143-185 kubenswrapper[2575]: W0730 07:13:02.701139 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d6eddcb_b229_4c8c_876c_78f88f269eef.slice/crio-92501ab0cb3195230c06d183a4d6aa32f487de15984ca26dba3af098ca9f5501 WatchSource:0}: Error finding container 92501ab0cb3195230c06d183a4d6aa32f487de15984ca26dba3af098ca9f5501: Status 404 returned error can't find the container with id 92501ab0cb3195230c06d183a4d6aa32f487de15984ca26dba3af098ca9f5501 Jul 30 07:13:03.274640 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:03.274598 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" event={"ID":"7d6eddcb-b229-4c8c-876c-78f88f269eef","Type":"ContainerStarted","Data":"1753f421eb84d0f6c309502e2524dea69033ab8b89dab12e04549c7936685b07"} Jul 30 07:13:03.274840 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:03.274691 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" event={"ID":"7d6eddcb-b229-4c8c-876c-78f88f269eef","Type":"ContainerStarted","Data":"92501ab0cb3195230c06d183a4d6aa32f487de15984ca26dba3af098ca9f5501"} Jul 30 07:13:08.293703 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:08.293642 2575 generic.go:356] "Generic (PLEG): container finished" podID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerID="1753f421eb84d0f6c309502e2524dea69033ab8b89dab12e04549c7936685b07" exitCode=0 Jul 30 07:13:08.294178 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:08.293715 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" event={"ID":"7d6eddcb-b229-4c8c-876c-78f88f269eef","Type":"ContainerDied","Data":"1753f421eb84d0f6c309502e2524dea69033ab8b89dab12e04549c7936685b07"} Jul 30 07:13:09.300527 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:09.300486 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" event={"ID":"7d6eddcb-b229-4c8c-876c-78f88f269eef","Type":"ContainerStarted","Data":"fd62e7ee0bc5dc7077f80b82756d5bef19ffc22c0389990ed3099a555dfcf5e1"} Jul 30 07:13:14.324122 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:14.324028 2575 generic.go:356] "Generic (PLEG): container finished" podID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerID="fd62e7ee0bc5dc7077f80b82756d5bef19ffc22c0389990ed3099a555dfcf5e1" exitCode=0 Jul 30 07:13:14.324122 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:14.324090 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" event={"ID":"7d6eddcb-b229-4c8c-876c-78f88f269eef","Type":"ContainerDied","Data":"fd62e7ee0bc5dc7077f80b82756d5bef19ffc22c0389990ed3099a555dfcf5e1"} Jul 30 07:13:15.334162 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:15.334110 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" event={"ID":"7d6eddcb-b229-4c8c-876c-78f88f269eef","Type":"ContainerStarted","Data":"9122845e89926878ac16c54f967919fa0cec5e5345856e53ca03289738f67e95"} Jul 30 07:13:15.334638 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:15.334173 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" event={"ID":"7d6eddcb-b229-4c8c-876c-78f88f269eef","Type":"ContainerStarted","Data":"26ce1e074f6a123fd948db21291bfa5d6331acc506a97ffe3bd0765203d5eb54"} Jul 30 07:13:15.334638 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:15.334211 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" event={"ID":"7d6eddcb-b229-4c8c-876c-78f88f269eef","Type":"ContainerStarted","Data":"fbb03b584ebbd43bc2f63f70cf1a89af33f86c8e8945f5c83e9ad054d258f3d2"} Jul 30 07:13:15.334638 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:15.334233 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" event={"ID":"7d6eddcb-b229-4c8c-876c-78f88f269eef","Type":"ContainerStarted","Data":"8860cd53e47d539ee4ffc8d9e9eded68b4e9b07d376f72356900ba7a812cbeb2"} Jul 30 07:13:15.364122 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:15.364050 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podStartSLOduration=14.364026046 podStartE2EDuration="14.364026046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 07:13:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 07:13:15.362190465 +0000 UTC m=+1756.719194633" watchObservedRunningTime="2026-07-30 07:13:15.364026046 +0000 UTC m=+1756.721030208" Jul 30 07:13:17.554395 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:17.554358 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:17.555019 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:17.554993 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:17.555164 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:17.555029 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:17.555164 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:17.555048 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:17.556738 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:17.556622 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.39:8443: connect: connection refused" Jul 30 07:13:17.556738 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:17.556648 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.39:8816: connect: connection refused" Jul 30 07:13:17.556738 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:17.556700 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused" Jul 30 07:13:17.556981 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:17.556763 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.39:6571: connect: connection refused" Jul 30 07:13:20.554902 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:20.554855 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.39:8816: connect: connection refused" Jul 30 07:13:20.555457 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:20.554854 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.39:8443: connect: connection refused" Jul 30 07:13:20.555884 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:20.555862 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:20.556432 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:20.556405 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:20.556813 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:20.556796 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:20.561757 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:20.561734 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:21.366181 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:21.366145 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:21.369389 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:21.369362 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:22.554526 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:22.554476 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:22.554526 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:22.554546 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:23.555396 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:23.555330 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:23.555805 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:23.555507 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:24.377618 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:24.377591 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:24.377808 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:24.377793 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:13:59.351347 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:59.351224 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/1.log" Jul 30 07:13:59.358419 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:59.356988 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/1.log" Jul 30 07:13:59.358419 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:59.358332 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 07:13:59.363939 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:13:59.363920 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 07:14:26.472358 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.472318 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq"] Jul 30 07:14:26.473389 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.473334 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="registry" containerID="cri-o://8860cd53e47d539ee4ffc8d9e9eded68b4e9b07d376f72356900ba7a812cbeb2" gracePeriod=30 Jul 30 07:14:26.473945 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.473918 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="offline" containerID="cri-o://26ce1e074f6a123fd948db21291bfa5d6331acc506a97ffe3bd0765203d5eb54" gracePeriod=30 Jul 30 07:14:26.474197 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.473964 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="ui" containerID="cri-o://9122845e89926878ac16c54f967919fa0cec5e5345856e53ca03289738f67e95" gracePeriod=30 Jul 30 07:14:26.474363 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.474034 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="online" containerID="cri-o://fbb03b584ebbd43bc2f63f70cf1a89af33f86c8e8945f5c83e9ad054d258f3d2" gracePeriod=30 Jul 30 07:14:26.772149 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.772061 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc"] Jul 30 07:14:26.776205 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.776182 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.778950 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.778924 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-nkhtr\"" Jul 30 07:14:26.790891 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.790864 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc"] Jul 30 07:14:26.848538 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.848491 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.848753 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.848616 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.848753 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.848678 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.848753 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.848727 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.848900 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.848756 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.848900 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.848786 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmxqr\" (UniqueName: \"kubernetes.io/projected/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-kube-api-access-wmxqr\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.949488 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.949452 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmxqr\" (UniqueName: \"kubernetes.io/projected/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-kube-api-access-wmxqr\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.949709 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.949502 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.949709 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.949547 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.949709 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.949567 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.949709 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.949585 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.949970 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.949718 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.949970 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:14:26.949728 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Jul 30 07:14:26.949970 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:14:26.949864 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-offline-tls podName:c42eda0c-71f1-4b9d-b58f-f6b43836a2e1 nodeName:}" failed. No retries permitted until 2026-07-30 07:14:27.449841947 +0000 UTC m=+1828.806846088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-offline-tls") pod "feast-simple-feast-setup-777d6cc998-5qpcc" (UID: "c42eda0c-71f1-4b9d-b58f-f6b43836a2e1") : secret "feast-simple-feast-setup-offline-tls" not found Jul 30 07:14:26.950163 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.950101 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.952496 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.952470 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.952778 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.952740 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.952778 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.952765 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:26.961019 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:26.960979 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmxqr\" (UniqueName: \"kubernetes.io/projected/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-kube-api-access-wmxqr\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:27.453989 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:27.453893 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:27.456453 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:27.456423 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-5qpcc\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:27.622973 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:27.622938 2575 generic.go:356] "Generic (PLEG): container finished" podID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerID="9122845e89926878ac16c54f967919fa0cec5e5345856e53ca03289738f67e95" exitCode=0 Jul 30 07:14:27.622973 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:27.622962 2575 generic.go:356] "Generic (PLEG): container finished" podID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerID="fbb03b584ebbd43bc2f63f70cf1a89af33f86c8e8945f5c83e9ad054d258f3d2" exitCode=0 Jul 30 07:14:27.622973 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:27.622981 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" event={"ID":"7d6eddcb-b229-4c8c-876c-78f88f269eef","Type":"ContainerDied","Data":"9122845e89926878ac16c54f967919fa0cec5e5345856e53ca03289738f67e95"} Jul 30 07:14:27.623423 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:27.623010 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" event={"ID":"7d6eddcb-b229-4c8c-876c-78f88f269eef","Type":"ContainerDied","Data":"fbb03b584ebbd43bc2f63f70cf1a89af33f86c8e8945f5c83e9ad054d258f3d2"} Jul 30 07:14:27.685858 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:27.685822 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:27.824740 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:27.824696 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc"] Jul 30 07:14:27.828620 ip-10-0-143-185 kubenswrapper[2575]: W0730 07:14:27.828585 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc42eda0c_71f1_4b9d_b58f_f6b43836a2e1.slice/crio-00c9f378bcac803fa04bc45d3a4d5452de28649d1e2c741c66609534ed9be589 WatchSource:0}: Error finding container 00c9f378bcac803fa04bc45d3a4d5452de28649d1e2c741c66609534ed9be589: Status 404 returned error can't find the container with id 00c9f378bcac803fa04bc45d3a4d5452de28649d1e2c741c66609534ed9be589 Jul 30 07:14:28.628438 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:28.628396 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" event={"ID":"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1","Type":"ContainerStarted","Data":"0bd8ae9f0a58b51bc4f75ea25d29573dbd4f660981f2bcb5c7e7b63416849e43"} Jul 30 07:14:28.628825 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:28.628457 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" event={"ID":"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1","Type":"ContainerStarted","Data":"00c9f378bcac803fa04bc45d3a4d5452de28649d1e2c741c66609534ed9be589"} Jul 30 07:14:31.366259 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:31.366220 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused" Jul 30 07:14:33.649451 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:33.649411 2575 generic.go:356] "Generic (PLEG): container finished" podID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerID="0bd8ae9f0a58b51bc4f75ea25d29573dbd4f660981f2bcb5c7e7b63416849e43" exitCode=0 Jul 30 07:14:33.649932 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:33.649481 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" event={"ID":"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1","Type":"ContainerDied","Data":"0bd8ae9f0a58b51bc4f75ea25d29573dbd4f660981f2bcb5c7e7b63416849e43"} Jul 30 07:14:34.378271 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:34.378225 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.39:8443: connect: connection refused" Jul 30 07:14:34.655711 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:34.655611 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" event={"ID":"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1","Type":"ContainerStarted","Data":"0aa718cfa937da388cf20d675920948dcc46950f706c54d8359d5827f7948b55"} Jul 30 07:14:39.677522 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:39.677484 2575 generic.go:356] "Generic (PLEG): container finished" podID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerID="0aa718cfa937da388cf20d675920948dcc46950f706c54d8359d5827f7948b55" exitCode=0 Jul 30 07:14:39.677913 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:39.677534 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" event={"ID":"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1","Type":"ContainerDied","Data":"0aa718cfa937da388cf20d675920948dcc46950f706c54d8359d5827f7948b55"} Jul 30 07:14:40.689053 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:40.688019 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" event={"ID":"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1","Type":"ContainerStarted","Data":"b25d6ca87c6e1ac10621a603cb01a877ae1303e85f0ab23248bdf03e9d51e59d"} Jul 30 07:14:40.689053 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:40.688077 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" event={"ID":"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1","Type":"ContainerStarted","Data":"cc85c57f9ec78a5c196960f4faba1859b2ed2d5cadcb810e81546e6a83aa43b2"} Jul 30 07:14:40.689053 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:40.688100 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" event={"ID":"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1","Type":"ContainerStarted","Data":"ee6f4ada30f3596ff3e6a5afa11710afc9f5124755a51f2c6567f96545e2a6ab"} Jul 30 07:14:40.689053 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:40.688118 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" event={"ID":"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1","Type":"ContainerStarted","Data":"d2d5dcb20b38c31f58ad1754a589dc464202a1c00546358b162c14a2168f174b"} Jul 30 07:14:40.716353 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:40.716224 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podStartSLOduration=14.716203756 podStartE2EDuration="14.716203756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 07:14:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 07:14:40.713472603 +0000 UTC m=+1842.070476770" watchObservedRunningTime="2026-07-30 07:14:40.716203756 +0000 UTC m=+1842.073207919" Jul 30 07:14:41.367132 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:41.367080 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused" Jul 30 07:14:42.686948 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:42.686909 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:42.687543 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:42.687516 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:42.687543 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:42.687550 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:42.687767 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:42.687567 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:42.689305 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:42.689268 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.40:6571: connect: connection refused" Jul 30 07:14:42.689426 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:42.689289 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused" Jul 30 07:14:42.689426 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:42.689321 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused" Jul 30 07:14:42.689426 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:42.689369 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.40:8816: connect: connection refused" Jul 30 07:14:44.378461 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:44.378404 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.39:8443: connect: connection refused" Jul 30 07:14:45.686258 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:45.686210 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused" Jul 30 07:14:45.686932 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:45.686908 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:45.687142 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:45.687124 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:45.687516 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:45.687493 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:45.687671 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:45.687640 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:45.687873 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:45.687851 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:45.687952 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:45.687923 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:45.726612 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:45.726579 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:46.727384 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:46.727353 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:46.730584 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:46.730561 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:47.686067 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:47.686026 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:48.686925 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:48.686894 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:48.734812 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:48.734781 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:14:51.366410 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:51.366364 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="online" probeResult="failure" output="Get \"https://10.132.0.39:6567/health\": dial tcp 10.132.0.39:6567: connect: connection refused" Jul 30 07:14:51.366911 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:51.366523 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:14:54.377714 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:54.377620 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.39:8443: connect: connection refused" Jul 30 07:14:54.378154 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:54.377832 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:14:56.770464 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:56.770381 2575 generic.go:356] "Generic (PLEG): container finished" podID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerID="26ce1e074f6a123fd948db21291bfa5d6331acc506a97ffe3bd0765203d5eb54" exitCode=137 Jul 30 07:14:56.770464 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:56.770410 2575 generic.go:356] "Generic (PLEG): container finished" podID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerID="8860cd53e47d539ee4ffc8d9e9eded68b4e9b07d376f72356900ba7a812cbeb2" exitCode=137 Jul 30 07:14:56.770464 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:56.770452 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" event={"ID":"7d6eddcb-b229-4c8c-876c-78f88f269eef","Type":"ContainerDied","Data":"26ce1e074f6a123fd948db21291bfa5d6331acc506a97ffe3bd0765203d5eb54"} Jul 30 07:14:56.770899 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:56.770493 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" event={"ID":"7d6eddcb-b229-4c8c-876c-78f88f269eef","Type":"ContainerDied","Data":"8860cd53e47d539ee4ffc8d9e9eded68b4e9b07d376f72356900ba7a812cbeb2"} Jul 30 07:14:57.130140 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.130117 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:14:57.222324 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.222292 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-offline-tls\") pod \"7d6eddcb-b229-4c8c-876c-78f88f269eef\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " Jul 30 07:14:57.222546 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.222343 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d6eddcb-b229-4c8c-876c-78f88f269eef-feast-data\") pod \"7d6eddcb-b229-4c8c-876c-78f88f269eef\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " Jul 30 07:14:57.222546 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.222369 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-registry-tls\") pod \"7d6eddcb-b229-4c8c-876c-78f88f269eef\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " Jul 30 07:14:57.222546 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.222400 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-online-tls\") pod \"7d6eddcb-b229-4c8c-876c-78f88f269eef\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " Jul 30 07:14:57.222546 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.222423 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqrxc\" (UniqueName: \"kubernetes.io/projected/7d6eddcb-b229-4c8c-876c-78f88f269eef-kube-api-access-mqrxc\") pod \"7d6eddcb-b229-4c8c-876c-78f88f269eef\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " Jul 30 07:14:57.222546 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.222445 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-ui-tls\") pod \"7d6eddcb-b229-4c8c-876c-78f88f269eef\" (UID: \"7d6eddcb-b229-4c8c-876c-78f88f269eef\") " Jul 30 07:14:57.222946 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.222904 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d6eddcb-b229-4c8c-876c-78f88f269eef-feast-data" (OuterVolumeSpecName: "feast-data") pod "7d6eddcb-b229-4c8c-876c-78f88f269eef" (UID: "7d6eddcb-b229-4c8c-876c-78f88f269eef"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 07:14:57.224942 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.224904 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d6eddcb-b229-4c8c-876c-78f88f269eef-kube-api-access-mqrxc" (OuterVolumeSpecName: "kube-api-access-mqrxc") pod "7d6eddcb-b229-4c8c-876c-78f88f269eef" (UID: "7d6eddcb-b229-4c8c-876c-78f88f269eef"). InnerVolumeSpecName "kube-api-access-mqrxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 07:14:57.225074 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.224963 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7d6eddcb-b229-4c8c-876c-78f88f269eef" (UID: "7d6eddcb-b229-4c8c-876c-78f88f269eef"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:14:57.225074 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.225001 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "7d6eddcb-b229-4c8c-876c-78f88f269eef" (UID: "7d6eddcb-b229-4c8c-876c-78f88f269eef"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:14:57.225167 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.225075 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-online-tls" (OuterVolumeSpecName: "online-tls") pod "7d6eddcb-b229-4c8c-876c-78f88f269eef" (UID: "7d6eddcb-b229-4c8c-876c-78f88f269eef"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:14:57.225167 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.225095 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "7d6eddcb-b229-4c8c-876c-78f88f269eef" (UID: "7d6eddcb-b229-4c8c-876c-78f88f269eef"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:14:57.323903 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.323806 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-online-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:14:57.323903 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.323844 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqrxc\" (UniqueName: \"kubernetes.io/projected/7d6eddcb-b229-4c8c-876c-78f88f269eef-kube-api-access-mqrxc\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:14:57.323903 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.323859 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-ui-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:14:57.323903 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.323871 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-offline-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:14:57.323903 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.323883 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7d6eddcb-b229-4c8c-876c-78f88f269eef-feast-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:14:57.323903 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.323894 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7d6eddcb-b229-4c8c-876c-78f88f269eef-registry-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:14:57.777703 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.777675 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" Jul 30 07:14:57.778085 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.777651 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq" event={"ID":"7d6eddcb-b229-4c8c-876c-78f88f269eef","Type":"ContainerDied","Data":"92501ab0cb3195230c06d183a4d6aa32f487de15984ca26dba3af098ca9f5501"} Jul 30 07:14:57.778085 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.777800 2575 scope.go:117] "RemoveContainer" containerID="9122845e89926878ac16c54f967919fa0cec5e5345856e53ca03289738f67e95" Jul 30 07:14:57.787100 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.787081 2575 scope.go:117] "RemoveContainer" containerID="26ce1e074f6a123fd948db21291bfa5d6331acc506a97ffe3bd0765203d5eb54" Jul 30 07:14:57.794714 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.794693 2575 scope.go:117] "RemoveContainer" containerID="fbb03b584ebbd43bc2f63f70cf1a89af33f86c8e8945f5c83e9ad054d258f3d2" Jul 30 07:14:57.801434 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.801411 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq"] Jul 30 07:14:57.802917 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.802900 2575 scope.go:117] "RemoveContainer" containerID="8860cd53e47d539ee4ffc8d9e9eded68b4e9b07d376f72356900ba7a812cbeb2" Jul 30 07:14:57.805363 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.805341 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-7m5hq"] Jul 30 07:14:57.810764 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.810746 2575 scope.go:117] "RemoveContainer" containerID="fd62e7ee0bc5dc7077f80b82756d5bef19ffc22c0389990ed3099a555dfcf5e1" Jul 30 07:14:57.818405 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:57.818386 2575 scope.go:117] "RemoveContainer" containerID="1753f421eb84d0f6c309502e2524dea69033ab8b89dab12e04549c7936685b07" Jul 30 07:14:59.176916 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:14:59.176886 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" path="/var/lib/kubelet/pods/7d6eddcb-b229-4c8c-876c-78f88f269eef/volumes" Jul 30 07:15:50.167497 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167416 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj"] Jul 30 07:15:50.167964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167723 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="registry" Jul 30 07:15:50.167964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167734 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="registry" Jul 30 07:15:50.167964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167743 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="offline" Jul 30 07:15:50.167964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167749 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="offline" Jul 30 07:15:50.167964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167770 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="feast-init" Jul 30 07:15:50.167964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167776 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="feast-init" Jul 30 07:15:50.167964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167783 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="online" Jul 30 07:15:50.167964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167788 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="online" Jul 30 07:15:50.167964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167800 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="ui" Jul 30 07:15:50.167964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167807 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="ui" Jul 30 07:15:50.167964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167813 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="feast-apply" Jul 30 07:15:50.167964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167818 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="feast-apply" Jul 30 07:15:50.167964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167867 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="offline" Jul 30 07:15:50.167964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167877 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="online" Jul 30 07:15:50.167964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167885 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="registry" Jul 30 07:15:50.167964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.167892 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d6eddcb-b229-4c8c-876c-78f88f269eef" containerName="ui" Jul 30 07:15:50.171531 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.171514 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.174186 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.174162 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jul 30 07:15:50.174324 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.174285 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jul 30 07:15:50.174324 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.174293 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jul 30 07:15:50.174440 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.174331 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-5zlhq\"" Jul 30 07:15:50.174488 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.174446 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jul 30 07:15:50.174527 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.174500 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jul 30 07:15:50.174527 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.174515 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jul 30 07:15:50.183376 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.183345 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj"] Jul 30 07:15:50.247680 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.247625 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.247861 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.247747 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.247861 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.247783 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl7b4\" (UniqueName: \"kubernetes.io/projected/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-kube-api-access-jl7b4\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.247861 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.247814 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.247861 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.247848 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.248061 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.247881 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.348565 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.348530 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.348565 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.348572 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.348847 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.348589 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl7b4\" (UniqueName: \"kubernetes.io/projected/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-kube-api-access-jl7b4\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.348847 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.348613 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.348847 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.348635 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.348847 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.348649 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.348847 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:15:50.348706 2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Jul 30 07:15:50.348847 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:15:50.348776 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-ui-tls podName:2e73d0fb-244e-4cb4-8d33-3dc11cb9be37 nodeName:}" failed. No retries permitted until 2026-07-30 07:15:50.848756752 +0000 UTC m=+1912.205760895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-ui-tls") pod "feast-simple-feast-remote-setup-db6955c6c-q7hwj" (UID: "2e73d0fb-244e-4cb4-8d33-3dc11cb9be37") : secret "feast-simple-feast-remote-setup-ui-tls" not found Jul 30 07:15:50.349120 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.349103 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.349435 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.349416 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.351285 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.351257 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.351408 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.351300 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.358083 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.358044 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl7b4\" (UniqueName: \"kubernetes.io/projected/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-kube-api-access-jl7b4\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.853716 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.853679 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:50.856345 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:50.856320 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-q7hwj\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:51.080432 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:51.080386 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:15:51.222994 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:51.222831 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj"] Jul 30 07:15:51.225896 ip-10-0-143-185 kubenswrapper[2575]: W0730 07:15:51.225867 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e73d0fb_244e_4cb4_8d33_3dc11cb9be37.slice/crio-c92d595e390456edcecd5c3bda683565c67b37d6310d11cabeebc73ff531f330 WatchSource:0}: Error finding container c92d595e390456edcecd5c3bda683565c67b37d6310d11cabeebc73ff531f330: Status 404 returned error can't find the container with id c92d595e390456edcecd5c3bda683565c67b37d6310d11cabeebc73ff531f330 Jul 30 07:15:51.982854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:51.982817 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" event={"ID":"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37","Type":"ContainerStarted","Data":"332dbe1581b18c8c23571ff0a5dbb4ad4375f77c887274a4da26f1c7318e5061"} Jul 30 07:15:51.982854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:51.982861 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" event={"ID":"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37","Type":"ContainerStarted","Data":"c92d595e390456edcecd5c3bda683565c67b37d6310d11cabeebc73ff531f330"} Jul 30 07:15:56.001182 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:56.001151 2575 generic.go:356] "Generic (PLEG): container finished" podID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerID="332dbe1581b18c8c23571ff0a5dbb4ad4375f77c887274a4da26f1c7318e5061" exitCode=0 Jul 30 07:15:56.001557 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:56.001194 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" event={"ID":"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37","Type":"ContainerDied","Data":"332dbe1581b18c8c23571ff0a5dbb4ad4375f77c887274a4da26f1c7318e5061"} Jul 30 07:15:57.007321 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:15:57.007286 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" event={"ID":"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37","Type":"ContainerStarted","Data":"ea6c591809836519229c7451b747e49209068bf4be969cb6d9d5f96a9edd6504"} Jul 30 07:16:02.032091 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:02.032059 2575 generic.go:356] "Generic (PLEG): container finished" podID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerID="ea6c591809836519229c7451b747e49209068bf4be969cb6d9d5f96a9edd6504" exitCode=0 Jul 30 07:16:02.032486 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:02.032132 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" event={"ID":"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37","Type":"ContainerDied","Data":"ea6c591809836519229c7451b747e49209068bf4be969cb6d9d5f96a9edd6504"} Jul 30 07:16:03.040013 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:03.039969 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" event={"ID":"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37","Type":"ContainerStarted","Data":"2d6805b99385bb9817cd43b5ae78e78723928f448087e4c008d315491b00c221"} Jul 30 07:16:03.040474 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:03.040021 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" event={"ID":"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37","Type":"ContainerStarted","Data":"a0e7191e1ef33f333ddff8526d77a0e845b2a141bc6b2325bb8a829ce1fc41cb"} Jul 30 07:16:03.040474 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:03.040055 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" event={"ID":"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37","Type":"ContainerStarted","Data":"196c5343a4473eb041019467a0c9b1213932d6303861ca71b1333887c918deed"} Jul 30 07:16:03.065140 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:03.065077 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podStartSLOduration=13.065056978 podStartE2EDuration="13.065056978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 07:15:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 07:16:03.063843611 +0000 UTC m=+1924.420847786" watchObservedRunningTime="2026-07-30 07:16:03.065056978 +0000 UTC m=+1924.422061142" Jul 30 07:16:03.081340 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:03.081282 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:16:03.081549 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:03.081357 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:16:03.081549 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:03.081377 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:16:03.082888 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:03.082851 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.41:8816: connect: connection refused" Jul 30 07:16:03.083029 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:03.082884 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused" Jul 30 07:16:03.083029 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:03.082913 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused" Jul 30 07:16:06.080971 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:06.080921 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.41:8816: connect: connection refused" Jul 30 07:16:06.080971 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:06.080942 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused" Jul 30 07:16:06.081512 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:06.080988 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused" Jul 30 07:16:09.081180 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:09.081138 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused" Jul 30 07:16:09.081940 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:09.081916 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:16:09.082408 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:09.082385 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:16:09.082798 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:09.082776 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:16:09.088427 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:09.088405 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:16:10.071505 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:10.071466 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:16:10.076081 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:10.076056 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:16:11.080636 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:11.080606 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:16:12.081804 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:12.081772 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:16:13.090244 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:16:13.090212 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:17:15.246684 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:15.246625 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj"] Jul 30 07:17:15.247190 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:15.246915 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="online" containerID="cri-o://196c5343a4473eb041019467a0c9b1213932d6303861ca71b1333887c918deed" gracePeriod=30 Jul 30 07:17:15.247190 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:15.247005 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="ui" containerID="cri-o://2d6805b99385bb9817cd43b5ae78e78723928f448087e4c008d315491b00c221" gracePeriod=30 Jul 30 07:17:15.247190 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:15.247032 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="offline" containerID="cri-o://a0e7191e1ef33f333ddff8526d77a0e845b2a141bc6b2325bb8a829ce1fc41cb" gracePeriod=30 Jul 30 07:17:15.362592 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:15.362560 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc"] Jul 30 07:17:15.362864 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:15.362830 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="registry" containerID="cri-o://d2d5dcb20b38c31f58ad1754a589dc464202a1c00546358b162c14a2168f174b" gracePeriod=30 Jul 30 07:17:15.362954 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:15.362869 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="ui" containerID="cri-o://b25d6ca87c6e1ac10621a603cb01a877ae1303e85f0ab23248bdf03e9d51e59d" gracePeriod=30 Jul 30 07:17:15.362954 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:15.362869 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="offline" containerID="cri-o://cc85c57f9ec78a5c196960f4faba1859b2ed2d5cadcb810e81546e6a83aa43b2" gracePeriod=30 Jul 30 07:17:15.362954 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:15.362868 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="online" containerID="cri-o://ee6f4ada30f3596ff3e6a5afa11710afc9f5124755a51f2c6567f96545e2a6ab" gracePeriod=30 Jul 30 07:17:16.350559 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:16.350481 2575 generic.go:356] "Generic (PLEG): container finished" podID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerID="2d6805b99385bb9817cd43b5ae78e78723928f448087e4c008d315491b00c221" exitCode=0 Jul 30 07:17:16.350559 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:16.350505 2575 generic.go:356] "Generic (PLEG): container finished" podID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerID="196c5343a4473eb041019467a0c9b1213932d6303861ca71b1333887c918deed" exitCode=0 Jul 30 07:17:16.350559 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:16.350550 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" event={"ID":"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37","Type":"ContainerDied","Data":"2d6805b99385bb9817cd43b5ae78e78723928f448087e4c008d315491b00c221"} Jul 30 07:17:16.351031 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:16.350593 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" event={"ID":"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37","Type":"ContainerDied","Data":"196c5343a4473eb041019467a0c9b1213932d6303861ca71b1333887c918deed"} Jul 30 07:17:16.353293 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:16.353266 2575 generic.go:356] "Generic (PLEG): container finished" podID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerID="b25d6ca87c6e1ac10621a603cb01a877ae1303e85f0ab23248bdf03e9d51e59d" exitCode=0 Jul 30 07:17:16.353293 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:16.353291 2575 generic.go:356] "Generic (PLEG): container finished" podID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerID="ee6f4ada30f3596ff3e6a5afa11710afc9f5124755a51f2c6567f96545e2a6ab" exitCode=0 Jul 30 07:17:16.353450 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:16.353325 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" event={"ID":"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1","Type":"ContainerDied","Data":"b25d6ca87c6e1ac10621a603cb01a877ae1303e85f0ab23248bdf03e9d51e59d"} Jul 30 07:17:16.353450 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:16.353360 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" event={"ID":"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1","Type":"ContainerDied","Data":"ee6f4ada30f3596ff3e6a5afa11710afc9f5124755a51f2c6567f96545e2a6ab"} Jul 30 07:17:16.728276 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:16.728228 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused" Jul 30 07:17:18.735012 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:18.734973 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused" Jul 30 07:17:20.072055 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:20.072012 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused" Jul 30 07:17:23.090502 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:23.090457 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused" Jul 30 07:17:26.728285 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:26.728238 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused" Jul 30 07:17:28.734937 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:28.734892 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused" Jul 30 07:17:30.071868 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:30.071824 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused" Jul 30 07:17:33.090397 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:33.090352 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused" Jul 30 07:17:36.727536 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:36.727491 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="online" probeResult="failure" output="Get \"https://10.132.0.40:6567/health\": dial tcp 10.132.0.40:6567: connect: connection refused" Jul 30 07:17:36.728017 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:36.727605 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:17:38.735179 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:38.735137 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.40:8443: connect: connection refused" Jul 30 07:17:38.735571 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:38.735273 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:17:40.072222 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:40.072182 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="online" probeResult="failure" output="Get \"https://10.132.0.41:6567/health\": dial tcp 10.132.0.41:6567: connect: connection refused" Jul 30 07:17:40.072715 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:40.072289 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:17:43.089891 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:43.089847 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.41:8443: connect: connection refused" Jul 30 07:17:43.090348 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:43.089969 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:17:45.467883 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.467855 2575 generic.go:356] "Generic (PLEG): container finished" podID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerID="a0e7191e1ef33f333ddff8526d77a0e845b2a141bc6b2325bb8a829ce1fc41cb" exitCode=137 Jul 30 07:17:45.468260 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.467926 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" event={"ID":"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37","Type":"ContainerDied","Data":"a0e7191e1ef33f333ddff8526d77a0e845b2a141bc6b2325bb8a829ce1fc41cb"} Jul 30 07:17:45.468260 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.467967 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" event={"ID":"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37","Type":"ContainerDied","Data":"c92d595e390456edcecd5c3bda683565c67b37d6310d11cabeebc73ff531f330"} Jul 30 07:17:45.468260 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.467984 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c92d595e390456edcecd5c3bda683565c67b37d6310d11cabeebc73ff531f330" Jul 30 07:17:45.469709 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.469687 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:17:45.470915 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.470895 2575 generic.go:356] "Generic (PLEG): container finished" podID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerID="cc85c57f9ec78a5c196960f4faba1859b2ed2d5cadcb810e81546e6a83aa43b2" exitCode=137 Jul 30 07:17:45.470915 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.470912 2575 generic.go:356] "Generic (PLEG): container finished" podID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerID="d2d5dcb20b38c31f58ad1754a589dc464202a1c00546358b162c14a2168f174b" exitCode=137 Jul 30 07:17:45.471054 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.470929 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" event={"ID":"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1","Type":"ContainerDied","Data":"cc85c57f9ec78a5c196960f4faba1859b2ed2d5cadcb810e81546e6a83aa43b2"} Jul 30 07:17:45.471054 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.470958 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" event={"ID":"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1","Type":"ContainerDied","Data":"d2d5dcb20b38c31f58ad1754a589dc464202a1c00546358b162c14a2168f174b"} Jul 30 07:17:45.513376 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.513292 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-registry-tls\") pod \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " Jul 30 07:17:45.513376 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.513342 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-online-tls\") pod \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " Jul 30 07:17:45.513376 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.513371 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jl7b4\" (UniqueName: \"kubernetes.io/projected/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-kube-api-access-jl7b4\") pod \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " Jul 30 07:17:45.513646 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.513426 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-ui-tls\") pod \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " Jul 30 07:17:45.513646 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.513450 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-feast-data\") pod \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " Jul 30 07:17:45.513646 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.513479 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-offline-tls\") pod \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\" (UID: \"2e73d0fb-244e-4cb4-8d33-3dc11cb9be37\") " Jul 30 07:17:45.513830 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.513780 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" (UID: "2e73d0fb-244e-4cb4-8d33-3dc11cb9be37"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 30 07:17:45.514113 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.514029 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-feast-data" (OuterVolumeSpecName: "feast-data") pod "2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" (UID: "2e73d0fb-244e-4cb4-8d33-3dc11cb9be37"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 07:17:45.515907 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.515877 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" (UID: "2e73d0fb-244e-4cb4-8d33-3dc11cb9be37"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:17:45.516013 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.515926 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-online-tls" (OuterVolumeSpecName: "online-tls") pod "2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" (UID: "2e73d0fb-244e-4cb4-8d33-3dc11cb9be37"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:17:45.516127 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.516106 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-kube-api-access-jl7b4" (OuterVolumeSpecName: "kube-api-access-jl7b4") pod "2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" (UID: "2e73d0fb-244e-4cb4-8d33-3dc11cb9be37"). InnerVolumeSpecName "kube-api-access-jl7b4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 07:17:45.516289 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.516269 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" (UID: "2e73d0fb-244e-4cb4-8d33-3dc11cb9be37"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:17:45.615088 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.615051 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-registry-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:17:45.615088 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.615082 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-online-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:17:45.615088 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.615093 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jl7b4\" (UniqueName: \"kubernetes.io/projected/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-kube-api-access-jl7b4\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:17:45.615312 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.615103 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-ui-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:17:45.615312 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.615112 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-feast-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:17:45.615312 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.615120 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37-offline-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:17:45.687827 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.687785 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.40:6571: connect: connection refused" Jul 30 07:17:45.687962 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:45.687785 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.40:8816: connect: connection refused" Jul 30 07:17:46.013364 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.013339 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:17:46.118964 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.118926 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-online-tls\") pod \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " Jul 30 07:17:46.119156 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.118981 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-feast-data\") pod \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " Jul 30 07:17:46.119156 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.119031 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-ui-tls\") pod \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " Jul 30 07:17:46.119156 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.119075 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmxqr\" (UniqueName: \"kubernetes.io/projected/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-kube-api-access-wmxqr\") pod \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " Jul 30 07:17:46.119156 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.119103 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-registry-tls\") pod \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " Jul 30 07:17:46.119156 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.119142 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-offline-tls\") pod \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\" (UID: \"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1\") " Jul 30 07:17:46.119615 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.119582 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-feast-data" (OuterVolumeSpecName: "feast-data") pod "c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" (UID: "c42eda0c-71f1-4b9d-b58f-f6b43836a2e1"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 07:17:46.121322 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.121294 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" (UID: "c42eda0c-71f1-4b9d-b58f-f6b43836a2e1"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:17:46.121428 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.121326 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-online-tls" (OuterVolumeSpecName: "online-tls") pod "c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" (UID: "c42eda0c-71f1-4b9d-b58f-f6b43836a2e1"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:17:46.121428 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.121348 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" (UID: "c42eda0c-71f1-4b9d-b58f-f6b43836a2e1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:17:46.121816 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.121799 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-kube-api-access-wmxqr" (OuterVolumeSpecName: "kube-api-access-wmxqr") pod "c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" (UID: "c42eda0c-71f1-4b9d-b58f-f6b43836a2e1"). InnerVolumeSpecName "kube-api-access-wmxqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 07:17:46.121885 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.121828 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" (UID: "c42eda0c-71f1-4b9d-b58f-f6b43836a2e1"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 30 07:17:46.220627 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.220533 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-offline-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:17:46.220627 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.220569 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-online-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:17:46.220627 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.220579 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-feast-data\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:17:46.220627 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.220587 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-ui-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:17:46.220627 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.220595 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmxqr\" (UniqueName: \"kubernetes.io/projected/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-kube-api-access-wmxqr\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:17:46.220627 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.220604 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1-registry-tls\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:17:46.477363 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.477278 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" Jul 30 07:17:46.477363 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.477286 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc" event={"ID":"c42eda0c-71f1-4b9d-b58f-f6b43836a2e1","Type":"ContainerDied","Data":"00c9f378bcac803fa04bc45d3a4d5452de28649d1e2c741c66609534ed9be589"} Jul 30 07:17:46.477363 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.477346 2575 scope.go:117] "RemoveContainer" containerID="b25d6ca87c6e1ac10621a603cb01a877ae1303e85f0ab23248bdf03e9d51e59d" Jul 30 07:17:46.477963 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.477574 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj" Jul 30 07:17:46.486204 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.486183 2575 scope.go:117] "RemoveContainer" containerID="cc85c57f9ec78a5c196960f4faba1859b2ed2d5cadcb810e81546e6a83aa43b2" Jul 30 07:17:46.497582 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.497565 2575 scope.go:117] "RemoveContainer" containerID="ee6f4ada30f3596ff3e6a5afa11710afc9f5124755a51f2c6567f96545e2a6ab" Jul 30 07:17:46.505172 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.505146 2575 scope.go:117] "RemoveContainer" containerID="d2d5dcb20b38c31f58ad1754a589dc464202a1c00546358b162c14a2168f174b" Jul 30 07:17:46.512996 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.512972 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc"] Jul 30 07:17:46.513490 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.513361 2575 scope.go:117] "RemoveContainer" containerID="0aa718cfa937da388cf20d675920948dcc46950f706c54d8359d5827f7948b55" Jul 30 07:17:46.515072 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.515055 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-5qpcc"] Jul 30 07:17:46.522992 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.522972 2575 scope.go:117] "RemoveContainer" containerID="0bd8ae9f0a58b51bc4f75ea25d29573dbd4f660981f2bcb5c7e7b63416849e43" Jul 30 07:17:46.529121 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.529097 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj"] Jul 30 07:17:46.535231 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:46.535209 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-q7hwj"] Jul 30 07:17:47.175401 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:47.175361 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" path="/var/lib/kubelet/pods/2e73d0fb-244e-4cb4-8d33-3dc11cb9be37/volumes" Jul 30 07:17:47.175926 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:17:47.175912 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" path="/var/lib/kubelet/pods/c42eda0c-71f1-4b9d-b58f-f6b43836a2e1/volumes" Jul 30 07:18:05.452359 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452321 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j5zgn/must-gather-mtw6n"] Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452595 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="online" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452605 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="online" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452620 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="online" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452625 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="online" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452633 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="ui" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452639 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="ui" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452647 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="feast-init" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452666 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="feast-init" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452673 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="registry" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452678 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="registry" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452688 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="offline" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452692 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="offline" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452698 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="ui" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452703 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="ui" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452710 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="feast-apply" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452716 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="feast-apply" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452721 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="offline" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452725 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="offline" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452733 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="feast-init" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452738 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="feast-init" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452747 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="feast-apply" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452752 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="feast-apply" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452793 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="online" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452801 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="offline" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452806 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="ui" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452811 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="registry" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452818 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c42eda0c-71f1-4b9d-b58f-f6b43836a2e1" containerName="ui" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452824 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="online" Jul 30 07:18:05.452854 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.452831 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e73d0fb-244e-4cb4-8d33-3dc11cb9be37" containerName="offline" Jul 30 07:18:05.455760 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.455739 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j5zgn/must-gather-mtw6n" Jul 30 07:18:05.458825 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.458805 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j5zgn\"/\"openshift-service-ca.crt\"" Jul 30 07:18:05.459254 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.459240 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j5zgn\"/\"kube-root-ca.crt\"" Jul 30 07:18:05.460058 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.460042 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-j5zgn\"/\"default-dockercfg-46nfw\"" Jul 30 07:18:05.465795 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.465773 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j5zgn/must-gather-mtw6n"] Jul 30 07:18:05.580989 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.580949 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpxcw\" (UniqueName: \"kubernetes.io/projected/04030c72-6da1-44cc-93d9-2246c15c55da-kube-api-access-qpxcw\") pod \"must-gather-mtw6n\" (UID: \"04030c72-6da1-44cc-93d9-2246c15c55da\") " pod="openshift-must-gather-j5zgn/must-gather-mtw6n" Jul 30 07:18:05.581167 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.581078 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/04030c72-6da1-44cc-93d9-2246c15c55da-must-gather-output\") pod \"must-gather-mtw6n\" (UID: \"04030c72-6da1-44cc-93d9-2246c15c55da\") " pod="openshift-must-gather-j5zgn/must-gather-mtw6n" Jul 30 07:18:05.682150 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.682114 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/04030c72-6da1-44cc-93d9-2246c15c55da-must-gather-output\") pod \"must-gather-mtw6n\" (UID: \"04030c72-6da1-44cc-93d9-2246c15c55da\") " pod="openshift-must-gather-j5zgn/must-gather-mtw6n" Jul 30 07:18:05.682321 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.682163 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpxcw\" (UniqueName: \"kubernetes.io/projected/04030c72-6da1-44cc-93d9-2246c15c55da-kube-api-access-qpxcw\") pod \"must-gather-mtw6n\" (UID: \"04030c72-6da1-44cc-93d9-2246c15c55da\") " pod="openshift-must-gather-j5zgn/must-gather-mtw6n" Jul 30 07:18:05.682458 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.682428 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/04030c72-6da1-44cc-93d9-2246c15c55da-must-gather-output\") pod \"must-gather-mtw6n\" (UID: \"04030c72-6da1-44cc-93d9-2246c15c55da\") " pod="openshift-must-gather-j5zgn/must-gather-mtw6n" Jul 30 07:18:05.691450 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.691421 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpxcw\" (UniqueName: \"kubernetes.io/projected/04030c72-6da1-44cc-93d9-2246c15c55da-kube-api-access-qpxcw\") pod \"must-gather-mtw6n\" (UID: \"04030c72-6da1-44cc-93d9-2246c15c55da\") " pod="openshift-must-gather-j5zgn/must-gather-mtw6n" Jul 30 07:18:05.777583 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.777483 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j5zgn/must-gather-mtw6n" Jul 30 07:18:05.911211 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.911180 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j5zgn/must-gather-mtw6n"] Jul 30 07:18:05.914297 ip-10-0-143-185 kubenswrapper[2575]: W0730 07:18:05.914271 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04030c72_6da1_44cc_93d9_2246c15c55da.slice/crio-91a626fc74e2e5e62af34ef1c659851ee69599e4842669c37a49e76089524d59 WatchSource:0}: Error finding container 91a626fc74e2e5e62af34ef1c659851ee69599e4842669c37a49e76089524d59: Status 404 returned error can't find the container with id 91a626fc74e2e5e62af34ef1c659851ee69599e4842669c37a49e76089524d59 Jul 30 07:18:05.916510 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:05.916490 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 30 07:18:06.555504 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:06.555465 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j5zgn/must-gather-mtw6n" event={"ID":"04030c72-6da1-44cc-93d9-2246c15c55da","Type":"ContainerStarted","Data":"91a626fc74e2e5e62af34ef1c659851ee69599e4842669c37a49e76089524d59"} Jul 30 07:18:12.585461 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:12.585357 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j5zgn/must-gather-mtw6n" event={"ID":"04030c72-6da1-44cc-93d9-2246c15c55da","Type":"ContainerStarted","Data":"0c7a949e996f0fab62a7fd27a42753ea1e536f28e5cf19f45df66285b9b65410"} Jul 30 07:18:12.585461 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:12.585413 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j5zgn/must-gather-mtw6n" event={"ID":"04030c72-6da1-44cc-93d9-2246c15c55da","Type":"ContainerStarted","Data":"433a1cb05743d74f683a6108d6ff12d61a5aacebbebbb5d4cf5dbc9303d73ea0"} Jul 30 07:18:12.606183 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:12.606131 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-j5zgn/must-gather-mtw6n" podStartSLOduration=1.350444511 podStartE2EDuration="7.606116629s" totalImagesPullingTime="6.255672118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 07:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 07:18:12.605356012 +0000 UTC m=+2053.962360172" watchObservedRunningTime="2026-07-30 07:18:12.606116629 +0000 UTC m=+2053.963120809" Jul 30 07:18:22.627864 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:22.627829 2575 generic.go:356] "Generic (PLEG): container finished" podID="04030c72-6da1-44cc-93d9-2246c15c55da" containerID="433a1cb05743d74f683a6108d6ff12d61a5aacebbebbb5d4cf5dbc9303d73ea0" exitCode=0 Jul 30 07:18:22.628294 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:22.627870 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j5zgn/must-gather-mtw6n" event={"ID":"04030c72-6da1-44cc-93d9-2246c15c55da","Type":"ContainerDied","Data":"433a1cb05743d74f683a6108d6ff12d61a5aacebbebbb5d4cf5dbc9303d73ea0"} Jul 30 07:18:22.628294 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:22.628198 2575 scope.go:117] "RemoveContainer" containerID="433a1cb05743d74f683a6108d6ff12d61a5aacebbebbb5d4cf5dbc9303d73ea0" Jul 30 07:18:23.047460 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:23.047381 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j5zgn_must-gather-mtw6n_04030c72-6da1-44cc-93d9-2246c15c55da/gather/0.log" Jul 30 07:18:23.532549 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:23.532512 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b5dgw/must-gather-wmshc"] Jul 30 07:18:23.534830 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:23.534816 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b5dgw/must-gather-wmshc" Jul 30 07:18:23.537586 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:23.537554 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b5dgw\"/\"openshift-service-ca.crt\"" Jul 30 07:18:23.537758 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:23.537731 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b5dgw\"/\"kube-root-ca.crt\"" Jul 30 07:18:23.537960 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:23.537945 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b5dgw\"/\"default-dockercfg-hk98j\"" Jul 30 07:18:23.545381 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:23.545360 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b5dgw/must-gather-wmshc"] Jul 30 07:18:23.627082 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:23.627052 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbpbm\" (UniqueName: \"kubernetes.io/projected/015de9a9-e9f1-40fd-91f2-4743cb67c353-kube-api-access-lbpbm\") pod \"must-gather-wmshc\" (UID: \"015de9a9-e9f1-40fd-91f2-4743cb67c353\") " pod="openshift-must-gather-b5dgw/must-gather-wmshc" Jul 30 07:18:23.627219 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:23.627093 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/015de9a9-e9f1-40fd-91f2-4743cb67c353-must-gather-output\") pod \"must-gather-wmshc\" (UID: \"015de9a9-e9f1-40fd-91f2-4743cb67c353\") " pod="openshift-must-gather-b5dgw/must-gather-wmshc" Jul 30 07:18:23.728148 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:23.728108 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/015de9a9-e9f1-40fd-91f2-4743cb67c353-must-gather-output\") pod \"must-gather-wmshc\" (UID: \"015de9a9-e9f1-40fd-91f2-4743cb67c353\") " pod="openshift-must-gather-b5dgw/must-gather-wmshc" Jul 30 07:18:23.728620 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:23.728186 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbpbm\" (UniqueName: \"kubernetes.io/projected/015de9a9-e9f1-40fd-91f2-4743cb67c353-kube-api-access-lbpbm\") pod \"must-gather-wmshc\" (UID: \"015de9a9-e9f1-40fd-91f2-4743cb67c353\") " pod="openshift-must-gather-b5dgw/must-gather-wmshc" Jul 30 07:18:23.728620 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:23.728532 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/015de9a9-e9f1-40fd-91f2-4743cb67c353-must-gather-output\") pod \"must-gather-wmshc\" (UID: \"015de9a9-e9f1-40fd-91f2-4743cb67c353\") " pod="openshift-must-gather-b5dgw/must-gather-wmshc" Jul 30 07:18:23.737375 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:23.737348 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbpbm\" (UniqueName: \"kubernetes.io/projected/015de9a9-e9f1-40fd-91f2-4743cb67c353-kube-api-access-lbpbm\") pod \"must-gather-wmshc\" (UID: \"015de9a9-e9f1-40fd-91f2-4743cb67c353\") " pod="openshift-must-gather-b5dgw/must-gather-wmshc" Jul 30 07:18:23.844852 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:23.844762 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b5dgw/must-gather-wmshc" Jul 30 07:18:23.979551 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:23.977263 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b5dgw/must-gather-wmshc"] Jul 30 07:18:23.982484 ip-10-0-143-185 kubenswrapper[2575]: W0730 07:18:23.982455 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod015de9a9_e9f1_40fd_91f2_4743cb67c353.slice/crio-5bc9c5f498dbc2dddd85eb52cc55f402fd920873ad0599bc93f76d5ce4cd51f9 WatchSource:0}: Error finding container 5bc9c5f498dbc2dddd85eb52cc55f402fd920873ad0599bc93f76d5ce4cd51f9: Status 404 returned error can't find the container with id 5bc9c5f498dbc2dddd85eb52cc55f402fd920873ad0599bc93f76d5ce4cd51f9 Jul 30 07:18:24.640010 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:24.639968 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b5dgw/must-gather-wmshc" event={"ID":"015de9a9-e9f1-40fd-91f2-4743cb67c353","Type":"ContainerStarted","Data":"5bc9c5f498dbc2dddd85eb52cc55f402fd920873ad0599bc93f76d5ce4cd51f9"} Jul 30 07:18:25.648717 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:25.648677 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b5dgw/must-gather-wmshc" event={"ID":"015de9a9-e9f1-40fd-91f2-4743cb67c353","Type":"ContainerStarted","Data":"2be0b09de028be1257c866a348fc17ce83f5a82dca7ba4c6b5d24526cc017813"} Jul 30 07:18:25.649139 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:25.648736 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b5dgw/must-gather-wmshc" event={"ID":"015de9a9-e9f1-40fd-91f2-4743cb67c353","Type":"ContainerStarted","Data":"365fe24feea5e87dcd00720733508c61679bf109977ed9bfa341edf2a80bacb2"} Jul 30 07:18:25.668427 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:25.668353 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b5dgw/must-gather-wmshc" podStartSLOduration=1.8752542399999999 podStartE2EDuration="2.668331594s" totalImagesPullingTime="793.077354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 07:18:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 07:18:25.667114692 +0000 UTC m=+2067.024118856" watchObservedRunningTime="2026-07-30 07:18:25.668331594 +0000 UTC m=+2067.025335758" Jul 30 07:18:26.112138 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:26.112017 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sc2rl_dd5d594e-9943-4d46-97e0-44cbbdc6ff6d/global-pull-secret-syncer/0.log" Jul 30 07:18:26.283707 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:26.283677 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hdqsd_7ce36c05-2aac-4717-8112-27dc9768ff02/konnectivity-agent/0.log" Jul 30 07:18:26.367302 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:26.367215 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-185.ec2.internal_93b23d5a25ebd708496785d7e0e39b30/haproxy/0.log" Jul 30 07:18:28.370858 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:28.370818 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-j5zgn/must-gather-mtw6n"] Jul 30 07:18:28.371320 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:28.371080 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-j5zgn/must-gather-mtw6n" podUID="04030c72-6da1-44cc-93d9-2246c15c55da" containerName="copy" containerID="cri-o://0c7a949e996f0fab62a7fd27a42753ea1e536f28e5cf19f45df66285b9b65410" gracePeriod=2 Jul 30 07:18:28.375254 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:28.375231 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-j5zgn/must-gather-mtw6n"] Jul 30 07:18:28.673601 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:28.673516 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j5zgn_must-gather-mtw6n_04030c72-6da1-44cc-93d9-2246c15c55da/copy/0.log" Jul 30 07:18:28.674331 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:28.674191 2575 generic.go:356] "Generic (PLEG): container finished" podID="04030c72-6da1-44cc-93d9-2246c15c55da" containerID="0c7a949e996f0fab62a7fd27a42753ea1e536f28e5cf19f45df66285b9b65410" exitCode=143 Jul 30 07:18:28.739319 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:28.739050 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j5zgn_must-gather-mtw6n_04030c72-6da1-44cc-93d9-2246c15c55da/copy/0.log" Jul 30 07:18:28.745534 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:28.742929 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j5zgn/must-gather-mtw6n" Jul 30 07:18:28.746149 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:18:28.746093 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-mtw6n\" is forbidden: User \"system:node:ip-10-0-143-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-j5zgn\": no relationship found between node 'ip-10-0-143-185.ec2.internal' and this object" podUID="04030c72-6da1-44cc-93d9-2246c15c55da" pod="openshift-must-gather-j5zgn/must-gather-mtw6n" Jul 30 07:18:28.774091 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:28.773053 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpxcw\" (UniqueName: \"kubernetes.io/projected/04030c72-6da1-44cc-93d9-2246c15c55da-kube-api-access-qpxcw\") pod \"04030c72-6da1-44cc-93d9-2246c15c55da\" (UID: \"04030c72-6da1-44cc-93d9-2246c15c55da\") " Jul 30 07:18:28.774091 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:28.773133 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/04030c72-6da1-44cc-93d9-2246c15c55da-must-gather-output\") pod \"04030c72-6da1-44cc-93d9-2246c15c55da\" (UID: \"04030c72-6da1-44cc-93d9-2246c15c55da\") " Jul 30 07:18:28.774091 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:28.773700 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04030c72-6da1-44cc-93d9-2246c15c55da-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "04030c72-6da1-44cc-93d9-2246c15c55da" (UID: "04030c72-6da1-44cc-93d9-2246c15c55da"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 30 07:18:28.784331 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:28.784283 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04030c72-6da1-44cc-93d9-2246c15c55da-kube-api-access-qpxcw" (OuterVolumeSpecName: "kube-api-access-qpxcw") pod "04030c72-6da1-44cc-93d9-2246c15c55da" (UID: "04030c72-6da1-44cc-93d9-2246c15c55da"). InnerVolumeSpecName "kube-api-access-qpxcw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 30 07:18:28.874006 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:28.873940 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpxcw\" (UniqueName: \"kubernetes.io/projected/04030c72-6da1-44cc-93d9-2246c15c55da-kube-api-access-qpxcw\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:18:28.874006 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:28.873979 2575 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/04030c72-6da1-44cc-93d9-2246c15c55da-must-gather-output\") on node \"ip-10-0-143-185.ec2.internal\" DevicePath \"\"" Jul 30 07:18:29.181185 ip-10-0-143-185 kubenswrapper[2575]: E0730 07:18:29.181144 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-mtw6n\" is forbidden: User \"system:node:ip-10-0-143-185.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-j5zgn\": no relationship found between node 'ip-10-0-143-185.ec2.internal' and this object" podUID="04030c72-6da1-44cc-93d9-2246c15c55da" pod="openshift-must-gather-j5zgn/must-gather-mtw6n" Jul 30 07:18:29.182515 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:29.182480 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04030c72-6da1-44cc-93d9-2246c15c55da" path="/var/lib/kubelet/pods/04030c72-6da1-44cc-93d9-2246c15c55da/volumes" Jul 30 07:18:29.700672 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:29.700172 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-j5zgn_must-gather-mtw6n_04030c72-6da1-44cc-93d9-2246c15c55da/copy/0.log" Jul 30 07:18:29.710991 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:29.710962 2575 scope.go:117] "RemoveContainer" containerID="0c7a949e996f0fab62a7fd27a42753ea1e536f28e5cf19f45df66285b9b65410" Jul 30 07:18:29.711329 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:29.711011 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j5zgn/must-gather-mtw6n" Jul 30 07:18:29.746905 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:29.746879 2575 scope.go:117] "RemoveContainer" containerID="433a1cb05743d74f683a6108d6ff12d61a5aacebbebbb5d4cf5dbc9303d73ea0" Jul 30 07:18:29.977537 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:29.977461 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ed5ae34f-c018-4cb2-90ad-bb172a6be9bd/alertmanager/0.log" Jul 30 07:18:30.019467 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:30.019435 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ed5ae34f-c018-4cb2-90ad-bb172a6be9bd/config-reloader/0.log" Jul 30 07:18:30.064585 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:30.064550 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ed5ae34f-c018-4cb2-90ad-bb172a6be9bd/kube-rbac-proxy-web/0.log" Jul 30 07:18:30.096941 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:30.096908 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ed5ae34f-c018-4cb2-90ad-bb172a6be9bd/kube-rbac-proxy/0.log" Jul 30 07:18:30.129398 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:30.129365 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ed5ae34f-c018-4cb2-90ad-bb172a6be9bd/kube-rbac-proxy-metric/0.log" Jul 30 07:18:30.157083 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:30.157049 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ed5ae34f-c018-4cb2-90ad-bb172a6be9bd/prom-label-proxy/0.log" Jul 30 07:18:30.185833 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:30.185808 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ed5ae34f-c018-4cb2-90ad-bb172a6be9bd/init-config-reloader/0.log" Jul 30 07:18:30.233760 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:30.233637 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-pptkm_b6eedc85-4962-46e8-a390-8e8547fbd0db/cluster-monitoring-operator/0.log" Jul 30 07:18:30.518570 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:30.518469 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gglzd_d05d0a5d-8ebf-4a60-88d4-69bb32d2f440/node-exporter/0.log" Jul 30 07:18:30.547166 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:30.547138 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gglzd_d05d0a5d-8ebf-4a60-88d4-69bb32d2f440/kube-rbac-proxy/0.log" Jul 30 07:18:30.579642 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:30.579617 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gglzd_d05d0a5d-8ebf-4a60-88d4-69bb32d2f440/init-textfile/0.log" Jul 30 07:18:31.070671 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:31.070622 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-7v92k_f24f984f-a413-416a-a941-876b3531376e/prometheus-operator/0.log" Jul 30 07:18:31.111949 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:31.111923 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-7v92k_f24f984f-a413-416a-a941-876b3531376e/kube-rbac-proxy/0.log" Jul 30 07:18:31.198096 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:31.198067 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76cb47f688-kspv5_0d44e3ab-79a3-4646-88d6-446242d7500d/telemeter-client/0.log" Jul 30 07:18:31.259455 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:31.259425 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76cb47f688-kspv5_0d44e3ab-79a3-4646-88d6-446242d7500d/reload/0.log" Jul 30 07:18:31.305300 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:31.305275 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76cb47f688-kspv5_0d44e3ab-79a3-4646-88d6-446242d7500d/kube-rbac-proxy/0.log" Jul 30 07:18:32.911324 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:32.911282 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-svjsc_52df3cd9-75cc-47c9-b997-48295db78d16/networking-console-plugin/0.log" Jul 30 07:18:33.410978 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.410952 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/1.log" Jul 30 07:18:33.416551 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.416524 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-794pk_0697e5d7-9959-4776-82b9-4a6f3e66b7c3/console-operator/2.log" Jul 30 07:18:33.835183 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.835084 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q"] Jul 30 07:18:33.835562 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.835541 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04030c72-6da1-44cc-93d9-2246c15c55da" containerName="gather" Jul 30 07:18:33.835633 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.835566 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="04030c72-6da1-44cc-93d9-2246c15c55da" containerName="gather" Jul 30 07:18:33.835633 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.835588 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04030c72-6da1-44cc-93d9-2246c15c55da" containerName="copy" Jul 30 07:18:33.835633 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.835597 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="04030c72-6da1-44cc-93d9-2246c15c55da" containerName="copy" Jul 30 07:18:33.835827 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.835703 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="04030c72-6da1-44cc-93d9-2246c15c55da" containerName="gather" Jul 30 07:18:33.835827 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.835719 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="04030c72-6da1-44cc-93d9-2246c15c55da" containerName="copy" Jul 30 07:18:33.839672 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.839627 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:33.855340 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.855303 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q"] Jul 30 07:18:33.927245 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.927215 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50ea2299-4531-41d6-a02e-80bc57ef1cd8-lib-modules\") pod \"perf-node-gather-daemonset-zjs2q\" (UID: \"50ea2299-4531-41d6-a02e-80bc57ef1cd8\") " pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:33.927705 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.927259 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/50ea2299-4531-41d6-a02e-80bc57ef1cd8-proc\") pod \"perf-node-gather-daemonset-zjs2q\" (UID: \"50ea2299-4531-41d6-a02e-80bc57ef1cd8\") " pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:33.927705 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.927278 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/50ea2299-4531-41d6-a02e-80bc57ef1cd8-podres\") pod \"perf-node-gather-daemonset-zjs2q\" (UID: \"50ea2299-4531-41d6-a02e-80bc57ef1cd8\") " pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:33.927705 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.927344 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d24pg\" (UniqueName: \"kubernetes.io/projected/50ea2299-4531-41d6-a02e-80bc57ef1cd8-kube-api-access-d24pg\") pod \"perf-node-gather-daemonset-zjs2q\" (UID: \"50ea2299-4531-41d6-a02e-80bc57ef1cd8\") " pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:33.927705 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.927451 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50ea2299-4531-41d6-a02e-80bc57ef1cd8-sys\") pod \"perf-node-gather-daemonset-zjs2q\" (UID: \"50ea2299-4531-41d6-a02e-80bc57ef1cd8\") " pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:33.974872 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:33.974843 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-gfn77_52145120-8982-4e8f-9b50-dcfa914cdb41/download-server/0.log" Jul 30 07:18:34.028492 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:34.028444 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/50ea2299-4531-41d6-a02e-80bc57ef1cd8-proc\") pod \"perf-node-gather-daemonset-zjs2q\" (UID: \"50ea2299-4531-41d6-a02e-80bc57ef1cd8\") " pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:34.028710 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:34.028592 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/50ea2299-4531-41d6-a02e-80bc57ef1cd8-proc\") pod \"perf-node-gather-daemonset-zjs2q\" (UID: \"50ea2299-4531-41d6-a02e-80bc57ef1cd8\") " pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:34.028710 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:34.028639 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/50ea2299-4531-41d6-a02e-80bc57ef1cd8-podres\") pod \"perf-node-gather-daemonset-zjs2q\" (UID: \"50ea2299-4531-41d6-a02e-80bc57ef1cd8\") " pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:34.028844 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:34.028817 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d24pg\" (UniqueName: \"kubernetes.io/projected/50ea2299-4531-41d6-a02e-80bc57ef1cd8-kube-api-access-d24pg\") pod \"perf-node-gather-daemonset-zjs2q\" (UID: \"50ea2299-4531-41d6-a02e-80bc57ef1cd8\") " pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:34.028902 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:34.028882 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/50ea2299-4531-41d6-a02e-80bc57ef1cd8-podres\") pod \"perf-node-gather-daemonset-zjs2q\" (UID: \"50ea2299-4531-41d6-a02e-80bc57ef1cd8\") " pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:34.028979 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:34.028960 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50ea2299-4531-41d6-a02e-80bc57ef1cd8-sys\") pod \"perf-node-gather-daemonset-zjs2q\" (UID: \"50ea2299-4531-41d6-a02e-80bc57ef1cd8\") " pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:34.029104 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:34.029068 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50ea2299-4531-41d6-a02e-80bc57ef1cd8-lib-modules\") pod \"perf-node-gather-daemonset-zjs2q\" (UID: \"50ea2299-4531-41d6-a02e-80bc57ef1cd8\") " pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:34.029242 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:34.029224 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/50ea2299-4531-41d6-a02e-80bc57ef1cd8-lib-modules\") pod \"perf-node-gather-daemonset-zjs2q\" (UID: \"50ea2299-4531-41d6-a02e-80bc57ef1cd8\") " pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:34.029314 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:34.029241 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/50ea2299-4531-41d6-a02e-80bc57ef1cd8-sys\") pod \"perf-node-gather-daemonset-zjs2q\" (UID: \"50ea2299-4531-41d6-a02e-80bc57ef1cd8\") " pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:34.040366 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:34.040332 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d24pg\" (UniqueName: \"kubernetes.io/projected/50ea2299-4531-41d6-a02e-80bc57ef1cd8-kube-api-access-d24pg\") pod \"perf-node-gather-daemonset-zjs2q\" (UID: \"50ea2299-4531-41d6-a02e-80bc57ef1cd8\") " pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:34.151855 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:34.151816 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:34.325606 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:34.325562 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q"] Jul 30 07:18:34.329962 ip-10-0-143-185 kubenswrapper[2575]: W0730 07:18:34.329932 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod50ea2299_4531_41d6_a02e_80bc57ef1cd8.slice/crio-b60e398d8ec2d85a1b9deecc1a20db90aad185a969215eef2a8c725551f6907e WatchSource:0}: Error finding container b60e398d8ec2d85a1b9deecc1a20db90aad185a969215eef2a8c725551f6907e: Status 404 returned error can't find the container with id b60e398d8ec2d85a1b9deecc1a20db90aad185a969215eef2a8c725551f6907e Jul 30 07:18:34.740169 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:34.740074 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" event={"ID":"50ea2299-4531-41d6-a02e-80bc57ef1cd8","Type":"ContainerStarted","Data":"95edccb96bdd1df21ee1f535f5d016e0feeeb96e224c8cbbfc9634de447e7ea0"} Jul 30 07:18:34.740169 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:34.740126 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" event={"ID":"50ea2299-4531-41d6-a02e-80bc57ef1cd8","Type":"ContainerStarted","Data":"b60e398d8ec2d85a1b9deecc1a20db90aad185a969215eef2a8c725551f6907e"} Jul 30 07:18:34.740418 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:34.740207 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:35.566976 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:35.566924 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pvswz_e85b5491-8587-430a-ac24-a541065ee2da/dns/0.log" Jul 30 07:18:35.597093 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:35.597069 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pvswz_e85b5491-8587-430a-ac24-a541065ee2da/kube-rbac-proxy/0.log" Jul 30 07:18:35.625007 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:35.624978 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7p8zb_2ba7155b-6db1-4d5e-aa46-a8cc4a2861f9/dns-node-resolver/0.log" Jul 30 07:18:36.246088 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:36.246061 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xmcq2_7adaf671-8982-4075-92cd-d190635a2fe7/node-ca/0.log" Jul 30 07:18:37.399408 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:37.399377 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bnkvv_510fa8de-adc5-476f-bb43-50233a5f291b/serve-healthcheck-canary/0.log" Jul 30 07:18:37.874745 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:37.874722 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-sflbp_bdbbfc08-6f67-4fda-b22a-4c00d352d582/insights-operator/0.log" Jul 30 07:18:37.874903 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:37.874820 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-sflbp_bdbbfc08-6f67-4fda-b22a-4c00d352d582/insights-operator/1.log" Jul 30 07:18:37.987480 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:37.987446 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7jxg6_e95e6dbb-61a0-4d2b-8600-591817523c32/kube-rbac-proxy/0.log" Jul 30 07:18:38.010888 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:38.010862 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7jxg6_e95e6dbb-61a0-4d2b-8600-591817523c32/exporter/0.log" Jul 30 07:18:38.038859 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:38.038830 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7jxg6_e95e6dbb-61a0-4d2b-8600-591817523c32/extractor/0.log" Jul 30 07:18:40.758971 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:40.758934 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" Jul 30 07:18:40.776762 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:40.776710 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b5dgw/perf-node-gather-daemonset-zjs2q" podStartSLOduration=7.776693551 podStartE2EDuration="7.776693551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 07:18:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 07:18:34.783358817 +0000 UTC m=+2076.140362980" watchObservedRunningTime="2026-07-30 07:18:40.776693551 +0000 UTC m=+2082.133697714" Jul 30 07:18:42.980414 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:42.980383 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-l9vxw_c167d2e1-1e5f-439a-9cc9-aed32eb9316a/migrator/0.log" Jul 30 07:18:43.017626 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:43.017600 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-l9vxw_c167d2e1-1e5f-439a-9cc9-aed32eb9316a/graceful-termination/0.log" Jul 30 07:18:44.519694 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:44.519645 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v7s2f_c57d113e-75be-4b93-92f6-4c28e8a4714e/kube-multus-additional-cni-plugins/0.log" Jul 30 07:18:44.546520 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:44.546492 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v7s2f_c57d113e-75be-4b93-92f6-4c28e8a4714e/egress-router-binary-copy/0.log" Jul 30 07:18:44.573253 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:44.573224 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v7s2f_c57d113e-75be-4b93-92f6-4c28e8a4714e/cni-plugins/0.log" Jul 30 07:18:44.600895 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:44.600870 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v7s2f_c57d113e-75be-4b93-92f6-4c28e8a4714e/bond-cni-plugin/0.log" Jul 30 07:18:44.627345 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:44.627317 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v7s2f_c57d113e-75be-4b93-92f6-4c28e8a4714e/routeoverride-cni/0.log" Jul 30 07:18:44.652835 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:44.652806 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v7s2f_c57d113e-75be-4b93-92f6-4c28e8a4714e/whereabouts-cni-bincopy/0.log" Jul 30 07:18:44.679102 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:44.679064 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v7s2f_c57d113e-75be-4b93-92f6-4c28e8a4714e/whereabouts-cni/0.log" Jul 30 07:18:44.958831 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:44.958804 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-h785z_961a0a68-4f71-40b4-926a-d9d3fda38987/kube-multus/0.log" Jul 30 07:18:45.086646 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:45.086546 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7tcdf_2a6fa524-6766-4ecd-a86a-5f26f29a72b2/network-metrics-daemon/0.log" Jul 30 07:18:45.132682 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:45.132631 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7tcdf_2a6fa524-6766-4ecd-a86a-5f26f29a72b2/kube-rbac-proxy/0.log" Jul 30 07:18:46.039736 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:46.039707 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-controller/0.log" Jul 30 07:18:46.090380 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:46.090354 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/0.log" Jul 30 07:18:46.090380 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:46.090359 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovn-acl-logging/1.log" Jul 30 07:18:46.134477 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:46.134451 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/kube-rbac-proxy-node/0.log" Jul 30 07:18:46.197848 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:46.197819 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/kube-rbac-proxy-ovn-metrics/0.log" Jul 30 07:18:46.247307 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:46.247275 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/northd/0.log" Jul 30 07:18:46.314554 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:46.314479 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/nbdb/0.log" Jul 30 07:18:46.355532 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:46.355505 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/sbdb/0.log" Jul 30 07:18:46.432069 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:46.432038 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84tq9_8e9d780d-93f9-40cd-a0f1-9577b8096849/ovnkube-controller/0.log" Jul 30 07:18:48.264986 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:48.264958 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-fr6kg_b5281138-d6b6-4c65-a18f-6b153031868e/network-check-target-container/0.log" Jul 30 07:18:49.282274 ip-10-0-143-185 kubenswrapper[2575]: I0730 07:18:49.282242 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-kkrfh_03374672-1c6c-4996-ae88-a5fa11c92a57/iptables-alerter/0.log"