Jul 22 16:11:42.891663 ip-10-0-129-160 systemd[1]: Starting Kubernetes Kubelet... Jul 22 16:11:43.286656 ip-10-0-129-160 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 16:11:43.286656 ip-10-0-129-160 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 16:11:43.286656 ip-10-0-129-160 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 16:11:43.286656 ip-10-0-129-160 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 16:11:43.286656 ip-10-0-129-160 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 16:11:43.287569 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.287491 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 16:11:43.294922 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294903 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:11:43.294922 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294920 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:11:43.294922 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294923 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:11:43.294922 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294926 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294929 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294932 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294937 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294939 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294942 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294945 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294947 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294950 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294952 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294955 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294958 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294960 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294963 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294965 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294969 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294971 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294974 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294976 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:11:43.295078 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294979 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294981 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294983 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294986 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294988 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294991 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294993 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.294997 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295001 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295004 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295010 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295012 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295015 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295017 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295020 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295023 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295026 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295029 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295031 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295033 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:11:43.295636 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295036 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295038 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295041 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295043 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295046 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295048 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295050 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295053 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295055 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295058 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295060 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295064 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295068 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295071 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295074 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295077 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295079 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295082 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295084 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295087 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:11:43.296146 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295090 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295092 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295094 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295097 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295099 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295102 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295105 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295108 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295110 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295113 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295115 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295118 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295120 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295124 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295128 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295131 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295133 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295136 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295139 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:11:43.296660 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295142 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295144 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295148 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295151 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295153 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295156 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295158 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295161 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295163 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295165 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295168 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295170 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295173 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295176 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295178 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295181 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295183 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295186 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295886 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295892 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:11:43.297144 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295895 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295898 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295901 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295903 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295906 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295908 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295911 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295913 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295916 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295918 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295921 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295923 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295925 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295930 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295934 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295937 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295940 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295943 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295946 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:11:43.297639 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295949 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295951 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295954 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295956 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295959 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295962 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295964 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295966 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295969 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295972 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295974 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295976 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295979 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295982 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295985 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295988 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295990 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295993 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295995 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.295998 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:11:43.298097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296001 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296003 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296005 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296008 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296010 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296013 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296016 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296018 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296020 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296023 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296026 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296029 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296032 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296034 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296037 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296041 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296044 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296047 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296050 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:11:43.298610 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296053 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296055 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296058 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296060 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296063 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296065 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296070 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296074 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296077 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296079 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296082 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296084 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296087 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296089 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296092 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296094 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296096 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296099 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296101 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:11:43.299062 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296104 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296106 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296108 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296111 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296114 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296117 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296119 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296123 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296125 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296132 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296135 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296137 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296140 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296142 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296144 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296147 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296149 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296152 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296155 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296161 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:11:43.299537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296517 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296523 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296526 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296529 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296531 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296534 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296536 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296539 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296541 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296544 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296547 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296550 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296552 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296555 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296557 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296560 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296563 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296565 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296568 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296570 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:11:43.300020 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296573 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296576 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296578 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296581 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296583 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296586 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296588 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296590 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296593 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296595 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296598 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296602 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296604 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296607 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296609 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296612 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296615 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296617 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296620 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296623 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:11:43.300540 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296625 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296628 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296630 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296632 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296635 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296637 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296640 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296642 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296644 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296647 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296650 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296652 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296655 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296657 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296660 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296662 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296665 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296667 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296670 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:11:43.301028 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296673 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296677 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296681 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296684 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296687 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296690 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296692 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296695 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296697 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296699 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296702 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296704 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296707 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296709 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296711 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296716 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296719 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296723 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296725 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:11:43.301563 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296728 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296730 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296733 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296735 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296738 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296740 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296742 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296745 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296747 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296749 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296752 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296754 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296757 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296761 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296763 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296766 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296768 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296771 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296773 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:11:43.302033 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296777 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.296780 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296850 2565 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296856 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296861 2565 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296866 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296869 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296873 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296877 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296881 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296884 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296886 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296890 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296893 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296896 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296899 2565 flags.go:64] FLAG: --cgroup-root="" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296902 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296905 2565 flags.go:64] FLAG: --client-ca-file="" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296908 2565 flags.go:64] FLAG: --cloud-provider="external" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296911 2565 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296914 2565 flags.go:64] FLAG: --cluster-domain="" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296917 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296920 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296923 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 16:11:43.302498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296926 2565 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296930 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296932 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296935 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296938 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296941 2565 flags.go:64] FLAG: --contention-profiling="false" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296944 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296947 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296952 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296955 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296958 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296961 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296965 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296967 2565 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296970 2565 flags.go:64] FLAG: --enable-server="true" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296973 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296976 2565 flags.go:64] FLAG: --event-burst="100" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296979 2565 flags.go:64] FLAG: --event-qps="50" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296982 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296985 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296988 2565 flags.go:64] FLAG: --eviction-hard="" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296991 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296994 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.296996 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297000 2565 flags.go:64] FLAG: --eviction-soft="" Jul 22 16:11:43.303095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297002 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297005 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297008 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297011 2565 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297013 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297016 2565 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297019 2565 flags.go:64] FLAG: --feature-gates="" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297022 2565 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297025 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297028 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297031 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297034 2565 flags.go:64] FLAG: --healthz-port="10248" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297036 2565 flags.go:64] FLAG: --help="false" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297039 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297042 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297046 2565 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297049 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297052 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297055 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297058 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297060 2565 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297063 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297065 2565 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 16:11:43.303725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297068 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297071 2565 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297073 2565 flags.go:64] FLAG: --kube-reserved="" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297076 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297078 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297081 2565 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297084 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297086 2565 flags.go:64] FLAG: --lock-file="" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297089 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297091 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297094 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297099 2565 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297102 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297105 2565 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297107 2565 flags.go:64] FLAG: --logging-format="text" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297110 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297113 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297115 2565 flags.go:64] FLAG: --manifest-url="" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297118 2565 flags.go:64] FLAG: --manifest-url-header="" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297122 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297124 2565 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297128 2565 flags.go:64] FLAG: --max-pods="110" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297132 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297135 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297139 2565 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 16:11:43.304312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297142 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297145 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297148 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297151 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297157 2565 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297160 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297162 2565 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297165 2565 flags.go:64] FLAG: --pod-cidr="" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297168 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297173 2565 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297175 2565 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297178 2565 flags.go:64] FLAG: --pods-per-core="0" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297181 2565 flags.go:64] FLAG: --port="10250" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297183 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297186 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fc2cda465e1dbb00" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297189 2565 flags.go:64] FLAG: --qos-reserved="" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297192 2565 flags.go:64] FLAG: --read-only-port="10255" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297195 2565 flags.go:64] FLAG: --register-node="true" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297197 2565 flags.go:64] FLAG: --register-with-taints="" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297200 2565 flags.go:64] FLAG: --registry-burst="10" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297203 2565 flags.go:64] FLAG: --registry-qps="5" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297206 2565 flags.go:64] FLAG: --reserved-cpus="" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297209 2565 flags.go:64] FLAG: --reserved-memory="" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297212 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297215 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 16:11:43.304930 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297217 2565 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297220 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297223 2565 flags.go:64] FLAG: --runonce="false" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297225 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297228 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297232 2565 flags.go:64] FLAG: --seccomp-default="false" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297236 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297239 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297242 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297245 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297248 2565 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297250 2565 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297253 2565 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297255 2565 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297258 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297261 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297264 2565 flags.go:64] FLAG: --system-cgroups="" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297266 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297271 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297274 2565 flags.go:64] FLAG: --tls-cert-file="" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297276 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297280 2565 flags.go:64] FLAG: --tls-min-version="" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297282 2565 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297285 2565 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297287 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 16:11:43.305556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297290 2565 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297293 2565 flags.go:64] FLAG: --v="2" Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297296 2565 flags.go:64] FLAG: --version="false" Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297300 2565 flags.go:64] FLAG: --vmodule="" Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297303 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.297306 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297424 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297429 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297432 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297435 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297438 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297440 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297445 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297449 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297451 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297454 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297456 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297459 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297461 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297463 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297466 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:11:43.306151 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297468 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297471 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297473 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297475 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297478 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297481 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297483 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297485 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297488 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297490 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297493 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297495 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297498 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297500 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297503 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297512 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297515 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297518 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297520 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297523 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:11:43.306764 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297525 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297527 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297530 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297534 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297538 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297541 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297543 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297546 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297548 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297552 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297555 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297558 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297561 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297564 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297566 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297569 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297571 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297574 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297576 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:11:43.307251 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297578 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297581 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297583 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297586 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297588 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297591 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297593 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297595 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297598 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297605 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297608 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297611 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297613 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297615 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297617 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297620 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297623 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297627 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297630 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297632 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:11:43.307728 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297634 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297637 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297640 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297642 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297645 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297647 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297650 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297653 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297655 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297657 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297660 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297662 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297665 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297667 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297671 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297674 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297677 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297679 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297688 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:11:43.308195 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297691 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:11:43.308655 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297694 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:11:43.308655 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297696 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:11:43.308655 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297704 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:11:43.308655 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297707 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:11:43.308655 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.297710 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:11:43.308655 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.298685 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 16:11:43.308655 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.305611 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 16:11:43.308655 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.305626 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 16:11:43.308655 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.305639 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 16:11:43.308655 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.305644 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 16:11:43.308655 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305700 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:11:43.308655 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305706 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:11:43.308655 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305709 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:11:43.308655 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305713 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305715 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305718 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305721 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305723 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305726 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305729 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305731 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305734 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305736 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305738 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305741 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305744 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305746 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305748 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305751 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305753 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305756 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305758 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:11:43.309006 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305760 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305763 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305766 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305768 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305771 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305773 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305775 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305779 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305781 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305784 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305786 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305788 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305791 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305793 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305796 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305798 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305801 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305803 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305805 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305808 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:11:43.309473 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305810 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305812 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305815 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305817 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305820 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305822 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305824 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305827 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305829 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305832 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305834 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305837 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305840 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305842 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305846 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305850 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305853 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305856 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305858 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305861 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:11:43.309959 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305864 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305867 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305870 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305872 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305875 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305877 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305880 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305882 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305884 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305887 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305890 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305893 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305895 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305897 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305900 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305902 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305905 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305907 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305909 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305912 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:11:43.310537 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305914 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:11:43.311001 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305916 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:11:43.311001 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305919 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:11:43.311001 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305921 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:11:43.311001 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305925 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:11:43.311001 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305928 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:11:43.311001 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305931 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:11:43.311001 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305934 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:11:43.311001 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305937 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:11:43.311001 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305940 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:11:43.311001 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305943 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:11:43.311001 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305945 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:11:43.311001 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305948 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:11:43.311001 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305951 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:11:43.311001 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305953 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:11:43.311001 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305956 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:11:43.311001 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.305959 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:11:43.311399 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.305964 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 16:11:43.311399 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306068 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 16:11:43.311399 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306072 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 16:11:43.311399 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306075 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 16:11:43.311399 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306078 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 16:11:43.311399 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306081 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 22 16:11:43.311399 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306083 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 16:11:43.311399 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306086 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 16:11:43.311399 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306089 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 16:11:43.311399 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306091 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 16:11:43.311399 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306094 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 16:11:43.311399 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306096 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 16:11:43.311399 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306098 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 16:11:43.311399 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306101 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 16:11:43.311399 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306103 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306106 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306108 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306110 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306113 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306115 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306118 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306120 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306122 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306125 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306128 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306131 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306133 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306136 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306138 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306141 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306143 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306145 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306148 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306150 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 16:11:43.311762 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306153 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306155 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306159 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306162 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306165 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306168 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306171 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306173 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306176 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306178 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306181 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306183 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306185 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306188 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306190 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306192 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306195 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306197 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306199 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 16:11:43.312254 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306202 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306204 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306207 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306209 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306212 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306214 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306216 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306219 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306221 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306223 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306226 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306228 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306230 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306234 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306237 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306239 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306242 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306245 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306248 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 16:11:43.312739 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306251 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306254 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306257 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306260 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306263 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306266 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306269 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306271 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306274 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306276 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306279 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306281 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306283 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306286 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306288 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306291 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306294 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306296 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306299 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306302 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 16:11:43.313186 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306304 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 16:11:43.313797 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306306 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 16:11:43.313797 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306309 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 16:11:43.313797 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306311 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 16:11:43.313797 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306314 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 16:11:43.313797 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306316 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 16:11:43.313797 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306318 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 16:11:43.313797 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:43.306320 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 16:11:43.313797 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.306325 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 16:11:43.313797 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.306427 2565 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 16:11:43.313797 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.308792 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 16:11:43.313797 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.310464 2565 server.go:1019] "Starting client certificate rotation" Jul 22 16:11:43.313797 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.310565 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 16:11:43.313797 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.311483 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 16:11:43.314170 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.314144 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 16:11:43.315175 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.315160 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 16:11:43.326894 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.326880 2565 log.go:25] "Validated CRI v1 runtime API" Jul 22 16:11:43.329563 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.329550 2565 log.go:25] "Validated CRI v1 image API" Jul 22 16:11:43.330341 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.330327 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 16:11:43.333859 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.333839 2565 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7ee3e1b4-66bf-4e2b-b6fc-6670d5e13e8e:/dev/nvme0n1p3 f5644544-f9b0-4444-bda9-097149b510e5:/dev/nvme0n1p4] Jul 22 16:11:43.333927 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.333858 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 16:11:43.338975 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.338860 2565 manager.go:216] Machine: {Timestamp:2026-07-22 16:11:43.337369344 +0000 UTC m=+0.350536374 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101926 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d32b188189528e8eba8f75f202745 SystemUUID:ec2d32b1-8818-9528-e8eb-a8f75f202745 BootID:d7a833e2-198a-4bf3-814e-eb19497027f1 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e1:92:7b:cb:af Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e1:92:7b:cb:af Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:61:fe:b3:f2:ca Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 16:11:43.338975 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.338970 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 16:11:43.339077 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.339061 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 16:11:43.342625 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.342599 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 16:11:43.342802 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.342627 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-160.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 16:11:43.343577 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.343566 2565 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 16:11:43.343612 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.343582 2565 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 16:11:43.343612 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.343591 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 16:11:43.345305 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.345290 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 16:11:43.346729 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.346714 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 16:11:43.347238 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.347226 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 16:11:43.351845 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.351834 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 22 16:11:43.351956 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.351946 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 16:11:43.351985 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.351977 2565 kubelet.go:486] "Attempting to sync node with API server" Jul 22 16:11:43.352011 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.351993 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 16:11:43.352044 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.352017 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 16:11:43.352044 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.352030 2565 kubelet.go:398] "Adding apiserver pod source" Jul 22 16:11:43.352103 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.352052 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 16:11:43.352374 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.352362 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 16:11:43.352431 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.352401 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 16:11:43.354716 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.354701 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 16:11:43.357067 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.357052 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 16:11:43.357121 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.357083 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 16:11:43.358597 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.358584 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 16:11:43.358667 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.358601 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 16:11:43.358667 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.358607 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 16:11:43.358667 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.358612 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 16:11:43.358667 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.358620 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 16:11:43.358667 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.358625 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 16:11:43.358667 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.358633 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 16:11:43.358667 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.358639 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 16:11:43.358667 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.358648 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 16:11:43.358667 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.358653 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 16:11:43.358667 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.358666 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 16:11:43.358667 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.358671 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 16:11:43.358977 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.358695 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 16:11:43.358977 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.358700 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 16:11:43.361804 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.361789 2565 server.go:1268] "Started kubelet" Jul 22 16:11:43.361978 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.361951 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 16:11:43.362403 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.362358 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 16:11:43.362511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.362470 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 16:11:43.362609 ip-10-0-129-160 systemd[1]: Started Kubernetes Kubelet. Jul 22 16:11:43.364552 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.364538 2565 server.go:310] "Adding debug handlers to kubelet server" Jul 22 16:11:43.364670 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.364638 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-160.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 16:11:43.364740 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.364715 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 16:11:43.364830 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.364789 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 16:11:43.366936 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.366911 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 16:11:43.368279 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.368257 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 16:11:43.371065 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.369948 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-160.ec2.internal.18c4a893657ab58c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-160.ec2.internal,UID:ip-10-0-129-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-160.ec2.internal,},FirstTimestamp:2026-07-22 16:11:43.361758604 +0000 UTC m=+0.374925636,LastTimestamp:2026-07-22 16:11:43.361758604 +0000 UTC m=+0.374925636,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-160.ec2.internal,}" Jul 22 16:11:43.376750 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.376737 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 16:11:43.376810 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.376760 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 16:11:43.377262 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.377220 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Jul 22 16:11:43.377478 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.377460 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 16:11:43.377478 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.377462 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 16:11:43.377606 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.377488 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 16:11:43.377606 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.377605 2565 reconstruct.go:97] "Volume reconstruction finished" Jul 22 16:11:43.377698 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.377612 2565 reconciler.go:29] "Reconciler: start to sync state" Jul 22 16:11:43.379036 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.379013 2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 22 16:11:43.379478 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.379423 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 22 16:11:43.381035 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.381020 2565 factory.go:55] Registering systemd factory Jul 22 16:11:43.381035 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.381035 2565 factory.go:223] Registration of the systemd container factory successfully Jul 22 16:11:43.381232 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.381218 2565 factory.go:153] Registering CRI-O factory Jul 22 16:11:43.381232 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.381232 2565 factory.go:223] Registration of the crio container factory successfully Jul 22 16:11:43.381334 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.381270 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 16:11:43.381334 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.381291 2565 factory.go:103] Registering Raw factory Jul 22 16:11:43.381334 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.381301 2565 manager.go:1196] Started watching for new ooms in manager Jul 22 16:11:43.382353 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.382337 2565 manager.go:349] Starting recovery of all containers Jul 22 16:11:43.392592 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.392571 2565 manager.go:354] Recovery completed Jul 22 16:11:43.396333 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.396321 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:11:43.398042 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.398018 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:11:43.398111 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.398053 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:11:43.398111 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.398067 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:11:43.398609 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.398593 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 16:11:43.398609 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.398608 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 16:11:43.398609 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.398610 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:11:43.398735 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.398623 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 22 16:11:43.399248 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.399232 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:11:43.399281 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.399266 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:11:43.399308 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.399281 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:11:43.400734 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.400721 2565 policy_none.go:49] "None policy: Start" Jul 22 16:11:43.400781 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.400737 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 16:11:43.400781 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.400747 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 16:11:43.400858 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.400841 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l252b" Jul 22 16:11:43.402700 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.402689 2565 policy_none.go:47] "Start" Jul 22 16:11:43.407751 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.407676 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-160.ec2.internal.18c4a89367a45527 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-160.ec2.internal,UID:ip-10-0-129-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-160.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-160.ec2.internal,},FirstTimestamp:2026-07-22 16:11:43.398040871 +0000 UTC m=+0.411207903,LastTimestamp:2026-07-22 16:11:43.398040871 +0000 UTC m=+0.411207903,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-160.ec2.internal,}" Jul 22 16:11:43.408976 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.408962 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l252b" Jul 22 16:11:43.435957 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.435942 2565 manager.go:337] "Starting Device Plugin manager" Jul 22 16:11:43.454289 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.435971 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 16:11:43.454289 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.435980 2565 server.go:87] "Starting device plugin registration server" Jul 22 16:11:43.454289 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.436199 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 16:11:43.454289 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.436327 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 16:11:43.454289 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.436442 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 16:11:43.454289 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.436508 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 16:11:43.454289 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.436514 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 16:11:43.454289 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.437632 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 16:11:43.454289 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.437674 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-160.ec2.internal\" not found" Jul 22 16:11:43.520553 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.520526 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 16:11:43.521836 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.521814 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 16:11:43.521901 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.521846 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 16:11:43.521901 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.521862 2565 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 16:11:43.521901 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.521891 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 16:11:43.524139 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.524121 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:11:43.537404 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.537349 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:11:43.538098 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.538083 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:11:43.538171 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.538108 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:11:43.538171 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.538118 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:11:43.538171 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.538136 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.551597 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.551581 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.551656 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.551600 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-160.ec2.internal\": node \"ip-10-0-129-160.ec2.internal\" not found" Jul 22 16:11:43.592659 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.592635 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Jul 22 16:11:43.622756 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.622734 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal"] Jul 22 16:11:43.622844 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.622799 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:11:43.623571 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.623555 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:11:43.623642 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.623581 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:11:43.623642 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.623590 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:11:43.625928 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.625916 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:11:43.626069 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.626041 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.626124 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.626082 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:11:43.626568 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.626552 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:11:43.626568 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.626559 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:11:43.626707 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.626577 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:11:43.626707 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.626587 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:11:43.626707 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.626586 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:11:43.626707 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.626659 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:11:43.628939 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.628923 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.629019 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.628951 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 16:11:43.629565 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.629548 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientMemory" Jul 22 16:11:43.629660 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.629574 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 16:11:43.629660 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.629585 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeHasSufficientPID" Jul 22 16:11:43.661039 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.661022 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-160.ec2.internal\" not found" node="ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.665515 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.665500 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-160.ec2.internal\" not found" node="ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.678839 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.678823 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f2872e03d19efdcaa9be42e12fa8b080-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal\" (UID: \"f2872e03d19efdcaa9be42e12fa8b080\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.678925 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.678845 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2872e03d19efdcaa9be42e12fa8b080-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal\" (UID: \"f2872e03d19efdcaa9be42e12fa8b080\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.678925 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.678862 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/db69a2dc3be3a098e4e8d4e246ebb928-config\") pod \"kube-apiserver-proxy-ip-10-0-129-160.ec2.internal\" (UID: \"db69a2dc3be3a098e4e8d4e246ebb928\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.692716 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.692688 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Jul 22 16:11:43.779761 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.779742 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/db69a2dc3be3a098e4e8d4e246ebb928-config\") pod \"kube-apiserver-proxy-ip-10-0-129-160.ec2.internal\" (UID: \"db69a2dc3be3a098e4e8d4e246ebb928\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.779821 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.779778 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/db69a2dc3be3a098e4e8d4e246ebb928-config\") pod \"kube-apiserver-proxy-ip-10-0-129-160.ec2.internal\" (UID: \"db69a2dc3be3a098e4e8d4e246ebb928\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.779821 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.779816 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f2872e03d19efdcaa9be42e12fa8b080-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal\" (UID: \"f2872e03d19efdcaa9be42e12fa8b080\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.779882 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.779834 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2872e03d19efdcaa9be42e12fa8b080-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal\" (UID: \"f2872e03d19efdcaa9be42e12fa8b080\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.779882 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.779865 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2872e03d19efdcaa9be42e12fa8b080-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal\" (UID: \"f2872e03d19efdcaa9be42e12fa8b080\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.779942 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.779893 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f2872e03d19efdcaa9be42e12fa8b080-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal\" (UID: \"f2872e03d19efdcaa9be42e12fa8b080\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.792802 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.792758 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Jul 22 16:11:43.893565 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.893537 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Jul 22 16:11:43.963745 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.963725 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.968249 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:43.968233 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" Jul 22 16:11:43.994205 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:43.994186 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Jul 22 16:11:44.094811 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:44.094754 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Jul 22 16:11:44.195242 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:44.195221 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Jul 22 16:11:44.295784 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:44.295763 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Jul 22 16:11:44.309929 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:44.309910 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 16:11:44.310062 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:44.310045 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 16:11:44.369630 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:44.369585 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:11:44.396180 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:44.396160 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-160.ec2.internal\" not found" Jul 22 16:11:44.415334 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:44.415307 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 16:06:43 +0000 UTC" deadline="2028-02-26 05:09:44.289403294 +0000 UTC" Jul 22 16:11:44.415334 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:44.415331 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14004h57m59.874074695s" Jul 22 16:11:44.435445 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:44.435429 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:11:44.477982 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:44.477962 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" Jul 22 16:11:44.496413 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:44.496376 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 16:11:44.497177 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:44.497162 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" Jul 22 16:11:44.509369 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:44.509355 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 16:11:44.519730 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:44.519716 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:11:44.565887 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:44.565860 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb69a2dc3be3a098e4e8d4e246ebb928.slice/crio-e42611a4e5d36ba8ffc535980949c16a854f6d140d119109e5afc3da358ccacb WatchSource:0}: Error finding container e42611a4e5d36ba8ffc535980949c16a854f6d140d119109e5afc3da358ccacb: Status 404 returned error can't find the container with id e42611a4e5d36ba8ffc535980949c16a854f6d140d119109e5afc3da358ccacb Jul 22 16:11:44.566809 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:44.566791 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2872e03d19efdcaa9be42e12fa8b080.slice/crio-f7f392b9273de842c7dc0d2f4ddb12b03e8cd3a18605c42fba9348987cbe60f5 WatchSource:0}: Error finding container f7f392b9273de842c7dc0d2f4ddb12b03e8cd3a18605c42fba9348987cbe60f5: Status 404 returned error can't find the container with id f7f392b9273de842c7dc0d2f4ddb12b03e8cd3a18605c42fba9348987cbe60f5 Jul 22 16:11:44.570616 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:44.570598 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 16:11:45.353470 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.353442 2565 apiserver.go:52] "Watching apiserver" Jul 22 16:11:45.363359 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.363334 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 16:11:45.364495 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.364465 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-blvd8","openshift-network-operator/iptables-alerter-z9cck","openshift-ovn-kubernetes/ovnkube-node-xjtvj","kube-system/global-pull-secret-syncer-jdctp","kube-system/konnectivity-agent-q44t2","openshift-cluster-node-tuning-operator/tuned-6lbnb","openshift-dns/node-resolver-st8vx","openshift-image-registry/node-ca-bpcxs","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal","openshift-multus/multus-vwml7","openshift-multus/network-metrics-daemon-gkw9w","kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll","openshift-multus/multus-additional-cni-plugins-bj4s6"] Jul 22 16:11:45.367268 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.367249 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:11:45.367363 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:45.367333 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:11:45.369994 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.369974 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z9cck" Jul 22 16:11:45.372408 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.372381 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.376887 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.373813 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:11:45.376887 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.374142 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bvfq9\"" Jul 22 16:11:45.376887 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.374324 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 16:11:45.376887 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.375796 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 16:11:45.378162 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.377646 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-flklf\"" Jul 22 16:11:45.378162 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.377702 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 16:11:45.378162 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.377757 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 16:11:45.378162 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.378001 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:45.378162 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.378008 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 16:11:45.378162 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:45.378072 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:11:45.378162 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.378141 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 16:11:45.378162 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.378163 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 16:11:45.378678 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.378180 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 16:11:45.378678 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.378183 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q44t2" Jul 22 16:11:45.378678 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.378296 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 16:11:45.380498 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.380476 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.381704 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.381653 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 16:11:45.382563 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.382177 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pnqcz\"" Jul 22 16:11:45.382563 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.382406 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 16:11:45.383275 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.383251 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-st8vx" Jul 22 16:11:45.383478 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.383438 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7q59s\"" Jul 22 16:11:45.383732 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.383717 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 16:11:45.383732 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.383725 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:11:45.386095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.386073 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bpcxs" Jul 22 16:11:45.386838 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.386816 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 16:11:45.386970 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.386822 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 16:11:45.387073 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.387052 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pnprs\"" Jul 22 16:11:45.387721 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.387693 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx72w\" (UniqueName: \"kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w\") pod \"network-check-target-blvd8\" (UID: \"7d94b09b-161f-4498-b0ac-9f3c0b104444\") " pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:11:45.387820 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.387740 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0097d54d-435a-471d-93f0-e9075cdec91c-iptables-alerter-script\") pod \"iptables-alerter-z9cck\" (UID: \"0097d54d-435a-471d-93f0-e9075cdec91c\") " pod="openshift-network-operator/iptables-alerter-z9cck" Jul 22 16:11:45.387820 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.387767 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/96df30a0-a2c9-42b3-9e2c-733801aa84b2-agent-certs\") pod \"konnectivity-agent-q44t2\" (UID: \"96df30a0-a2c9-42b3-9e2c-733801aa84b2\") " pod="kube-system/konnectivity-agent-q44t2" Jul 22 16:11:45.387820 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.387797 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-kubelet\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.387990 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.387824 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-var-lib-openvswitch\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.387990 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.387850 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/29719853-4c4b-4f4f-bcc3-2cc539e6da52-env-overrides\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.387990 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.387880 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/96df30a0-a2c9-42b3-9e2c-733801aa84b2-konnectivity-ca\") pod \"konnectivity-agent-q44t2\" (UID: \"96df30a0-a2c9-42b3-9e2c-733801aa84b2\") " pod="kube-system/konnectivity-agent-q44t2" Jul 22 16:11:45.387990 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.387906 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-run-openvswitch\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.387990 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.387943 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret\") pod \"global-pull-secret-syncer-jdctp\" (UID: \"6b58efe5-eede-4fbf-a506-ae944342040f\") " pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:45.387990 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.387972 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-run-ovn-kubernetes\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.387990 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.387988 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.388267 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388006 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/29719853-4c4b-4f4f-bcc3-2cc539e6da52-ovnkube-script-lib\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.388267 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388030 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-run-systemd\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.388267 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388053 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-run-ovn\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.388267 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388077 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-cni-bin\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.388267 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388101 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-cni-netd\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.388267 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388126 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6b58efe5-eede-4fbf-a506-ae944342040f-dbus\") pod \"global-pull-secret-syncer-jdctp\" (UID: \"6b58efe5-eede-4fbf-a506-ae944342040f\") " pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:45.388267 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388193 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/29719853-4c4b-4f4f-bcc3-2cc539e6da52-ovnkube-config\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.388267 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388223 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0097d54d-435a-471d-93f0-e9075cdec91c-host-slash\") pod \"iptables-alerter-z9cck\" (UID: \"0097d54d-435a-471d-93f0-e9075cdec91c\") " pod="openshift-network-operator/iptables-alerter-z9cck" Jul 22 16:11:45.388267 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388263 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-systemd-units\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.388675 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388291 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-etc-openvswitch\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.388675 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388318 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-node-log\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.388675 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388344 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/29719853-4c4b-4f4f-bcc3-2cc539e6da52-ovn-node-metrics-cert\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.388675 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388382 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbb2d\" (UniqueName: \"kubernetes.io/projected/0097d54d-435a-471d-93f0-e9075cdec91c-kube-api-access-mbb2d\") pod \"iptables-alerter-z9cck\" (UID: \"0097d54d-435a-471d-93f0-e9075cdec91c\") " pod="openshift-network-operator/iptables-alerter-z9cck" Jul 22 16:11:45.388675 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388413 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-slash\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.388675 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388441 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-run-netns\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.388675 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388533 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-log-socket\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.388675 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388577 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdtqt\" (UniqueName: \"kubernetes.io/projected/29719853-4c4b-4f4f-bcc3-2cc539e6da52-kube-api-access-tdtqt\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.388675 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388655 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6b58efe5-eede-4fbf-a506-ae944342040f-kubelet-config\") pod \"global-pull-secret-syncer-jdctp\" (UID: \"6b58efe5-eede-4fbf-a506-ae944342040f\") " pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:45.389200 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.388821 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.389200 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.389101 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 16:11:45.389364 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.389348 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 16:11:45.389601 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.389568 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 16:11:45.389681 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.389667 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zhzrz\"" Jul 22 16:11:45.392560 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.391306 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:45.392560 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:45.391387 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:11:45.394169 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.394148 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 16:11:45.394479 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.394445 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.394479 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.394472 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 16:11:45.395313 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.394910 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 16:11:45.395313 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.395114 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 16:11:45.395313 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.395254 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 16:11:45.395541 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.395519 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-s7x77\"" Jul 22 16:11:45.396716 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.396693 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.397921 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.397902 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6779r\"" Jul 22 16:11:45.398164 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.397908 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 16:11:45.398275 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.398081 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 16:11:45.398341 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.398117 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 16:11:45.400282 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.399502 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lbpfp\"" Jul 22 16:11:45.400282 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.399774 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 16:11:45.400282 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.400035 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 16:11:45.444463 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.444435 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s85wp" Jul 22 16:11:45.459431 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.459120 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s85wp" Jul 22 16:11:45.478304 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.478283 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 16:11:45.488866 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.488841 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx72w\" (UniqueName: \"kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w\") pod \"network-check-target-blvd8\" (UID: \"7d94b09b-161f-4498-b0ac-9f3c0b104444\") " pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:11:45.488971 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.488885 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.488971 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.488912 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/592b0f74-1d67-4361-85ba-6fe61394a87e-cni-binary-copy\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.488971 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.488938 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/592b0f74-1d67-4361-85ba-6fe61394a87e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.489143 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.488967 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgh4m\" (UniqueName: \"kubernetes.io/projected/6aae32f3-293e-4611-b044-892e51135c16-kube-api-access-tgh4m\") pod \"node-resolver-st8vx\" (UID: \"6aae32f3-293e-4611-b044-892e51135c16\") " pod="openshift-dns/node-resolver-st8vx" Jul 22 16:11:45.489143 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.488998 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-run-netns\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.489143 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489029 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-var-lib-openvswitch\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.489143 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489055 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/592b0f74-1d67-4361-85ba-6fe61394a87e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.489143 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489080 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/592b0f74-1d67-4361-85ba-6fe61394a87e-os-release\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.489143 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489110 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/96df30a0-a2c9-42b3-9e2c-733801aa84b2-konnectivity-ca\") pod \"konnectivity-agent-q44t2\" (UID: \"96df30a0-a2c9-42b3-9e2c-733801aa84b2\") " pod="kube-system/konnectivity-agent-q44t2" Jul 22 16:11:45.489143 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489134 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-run-openvswitch\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.489509 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489159 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret\") pod \"global-pull-secret-syncer-jdctp\" (UID: \"6b58efe5-eede-4fbf-a506-ae944342040f\") " pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:45.489509 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489183 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-cnibin\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.489509 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489208 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-var-lib-cni-bin\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.489509 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489235 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-run-ovn-kubernetes\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.489509 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489261 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.489509 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489287 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/29719853-4c4b-4f4f-bcc3-2cc539e6da52-ovnkube-script-lib\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.489509 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489317 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn4cg\" (UniqueName: \"kubernetes.io/projected/536aeccf-be47-44e1-a65c-572c81ba240a-kube-api-access-zn4cg\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.489509 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489342 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzfjs\" (UniqueName: \"kubernetes.io/projected/592b0f74-1d67-4361-85ba-6fe61394a87e-kube-api-access-jzfjs\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.489509 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489366 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-etc-kubernetes\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.489509 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489391 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b8dd\" (UniqueName: \"kubernetes.io/projected/9fafd1b8-f815-4a95-982f-fe79e9086034-kube-api-access-9b8dd\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.489509 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489416 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-cni-bin\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.489509 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489463 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-cni-netd\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.490117 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489561 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-run-openvswitch\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.490117 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.489639 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-var-lib-openvswitch\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.493644 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490237 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/96df30a0-a2c9-42b3-9e2c-733801aa84b2-konnectivity-ca\") pod \"konnectivity-agent-q44t2\" (UID: \"96df30a0-a2c9-42b3-9e2c-733801aa84b2\") " pod="kube-system/konnectivity-agent-q44t2" Jul 22 16:11:45.493644 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490385 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-run-ovn-kubernetes\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.493644 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490411 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/29719853-4c4b-4f4f-bcc3-2cc539e6da52-ovnkube-script-lib\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.493644 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:45.490416 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:11:45.493644 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490484 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.493644 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490493 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-cni-bin\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.493644 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:45.490553 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret podName:6b58efe5-eede-4fbf-a506-ae944342040f nodeName:}" failed. No retries permitted until 2026-07-22 16:11:45.990527489 +0000 UTC m=+3.003694523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret") pod "global-pull-secret-syncer-jdctp" (UID: "6b58efe5-eede-4fbf-a506-ae944342040f") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:11:45.493644 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490576 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6b58efe5-eede-4fbf-a506-ae944342040f-dbus\") pod \"global-pull-secret-syncer-jdctp\" (UID: \"6b58efe5-eede-4fbf-a506-ae944342040f\") " pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:45.493644 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490590 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-cni-netd\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.493644 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490606 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-sysctl-conf\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.493644 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490626 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdnfk\" (UniqueName: \"kubernetes.io/projected/3564cf28-82d3-4856-8ee0-17c61398d176-kube-api-access-gdnfk\") pod \"node-ca-bpcxs\" (UID: \"3564cf28-82d3-4856-8ee0-17c61398d176\") " pod="openshift-image-registry/node-ca-bpcxs" Jul 22 16:11:45.493644 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490662 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-run-multus-certs\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.493644 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490689 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-sysctl-d\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.493644 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490720 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-device-dir\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.493644 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490766 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/592b0f74-1d67-4361-85ba-6fe61394a87e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.493644 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490794 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9fafd1b8-f815-4a95-982f-fe79e9086034-cni-binary-copy\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490804 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6b58efe5-eede-4fbf-a506-ae944342040f-dbus\") pod \"global-pull-secret-syncer-jdctp\" (UID: \"6b58efe5-eede-4fbf-a506-ae944342040f\") " pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490836 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9fafd1b8-f815-4a95-982f-fe79e9086034-multus-daemon-config\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490862 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6l5b\" (UniqueName: \"kubernetes.io/projected/3dc7600c-4705-48d3-a410-eebfaa40c9cf-kube-api-access-f6l5b\") pod \"network-metrics-daemon-gkw9w\" (UID: \"3dc7600c-4705-48d3-a410-eebfaa40c9cf\") " pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490906 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbb2d\" (UniqueName: \"kubernetes.io/projected/0097d54d-435a-471d-93f0-e9075cdec91c-kube-api-access-mbb2d\") pod \"iptables-alerter-z9cck\" (UID: \"0097d54d-435a-471d-93f0-e9075cdec91c\") " pod="openshift-network-operator/iptables-alerter-z9cck" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490937 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-slash\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.490984 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-slash\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491072 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-run-netns\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491088 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6b58efe5-eede-4fbf-a506-ae944342040f-kubelet-config\") pod \"global-pull-secret-syncer-jdctp\" (UID: \"6b58efe5-eede-4fbf-a506-ae944342040f\") " pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491122 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-sys\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491137 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-multus-cni-dir\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491153 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0097d54d-435a-471d-93f0-e9075cdec91c-iptables-alerter-script\") pod \"iptables-alerter-z9cck\" (UID: \"0097d54d-435a-471d-93f0-e9075cdec91c\") " pod="openshift-network-operator/iptables-alerter-z9cck" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491167 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-sysconfig\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491166 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-run-netns\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491183 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-kubernetes\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491199 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/592b0f74-1d67-4361-85ba-6fe61394a87e-system-cni-dir\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491217 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3564cf28-82d3-4856-8ee0-17c61398d176-host\") pod \"node-ca-bpcxs\" (UID: \"3564cf28-82d3-4856-8ee0-17c61398d176\") " pod="openshift-image-registry/node-ca-bpcxs" Jul 22 16:11:45.494638 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491232 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-multus-socket-dir-parent\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491234 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6b58efe5-eede-4fbf-a506-ae944342040f-kubelet-config\") pod \"global-pull-secret-syncer-jdctp\" (UID: \"6b58efe5-eede-4fbf-a506-ae944342040f\") " pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491246 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/96df30a0-a2c9-42b3-9e2c-733801aa84b2-agent-certs\") pod \"konnectivity-agent-q44t2\" (UID: \"96df30a0-a2c9-42b3-9e2c-733801aa84b2\") " pod="kube-system/konnectivity-agent-q44t2" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491270 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-kubelet\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491298 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/29719853-4c4b-4f4f-bcc3-2cc539e6da52-env-overrides\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491325 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6aae32f3-293e-4611-b044-892e51135c16-tmp-dir\") pod \"node-resolver-st8vx\" (UID: \"6aae32f3-293e-4611-b044-892e51135c16\") " pod="openshift-dns/node-resolver-st8vx" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491348 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-os-release\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491373 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/592b0f74-1d67-4361-85ba-6fe61394a87e-cnibin\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491409 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-run\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491452 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/536aeccf-be47-44e1-a65c-572c81ba240a-etc-tuned\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491477 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-socket-dir\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491503 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-registration-dir\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491531 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-system-cni-dir\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491557 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-var-lib-kubelet\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491587 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs\") pod \"network-metrics-daemon-gkw9w\" (UID: \"3dc7600c-4705-48d3-a410-eebfaa40c9cf\") " pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491610 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-systemd\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491633 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-lib-modules\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.495511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491657 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/536aeccf-be47-44e1-a65c-572c81ba240a-tmp\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491684 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-hostroot\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491708 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-multus-conf-dir\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491735 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-run-systemd\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491789 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-run-ovn\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491814 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-host\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491841 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-sys-fs\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491869 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-run-k8s-cni-cncf-io\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491897 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/29719853-4c4b-4f4f-bcc3-2cc539e6da52-ovnkube-config\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491927 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-modprobe-d\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491949 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0097d54d-435a-471d-93f0-e9075cdec91c-iptables-alerter-script\") pod \"iptables-alerter-z9cck\" (UID: \"0097d54d-435a-471d-93f0-e9075cdec91c\") " pod="openshift-network-operator/iptables-alerter-z9cck" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491969 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-var-lib-kubelet\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.491529 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492014 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492080 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-host-kubelet\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492451 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/29719853-4c4b-4f4f-bcc3-2cc539e6da52-env-overrides\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492514 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6aae32f3-293e-4611-b044-892e51135c16-hosts-file\") pod \"node-resolver-st8vx\" (UID: \"6aae32f3-293e-4611-b044-892e51135c16\") " pod="openshift-dns/node-resolver-st8vx" Jul 22 16:11:45.496367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492538 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-run-ovn\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492545 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3564cf28-82d3-4856-8ee0-17c61398d176-serviceca\") pod \"node-ca-bpcxs\" (UID: \"3564cf28-82d3-4856-8ee0-17c61398d176\") " pod="openshift-image-registry/node-ca-bpcxs" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492589 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0097d54d-435a-471d-93f0-e9075cdec91c-host-slash\") pod \"iptables-alerter-z9cck\" (UID: \"0097d54d-435a-471d-93f0-e9075cdec91c\") " pod="openshift-network-operator/iptables-alerter-z9cck" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492630 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0097d54d-435a-471d-93f0-e9075cdec91c-host-slash\") pod \"iptables-alerter-z9cck\" (UID: \"0097d54d-435a-471d-93f0-e9075cdec91c\") " pod="openshift-network-operator/iptables-alerter-z9cck" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492678 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-systemd-units\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492712 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-etc-openvswitch\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492738 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-node-log\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492776 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/29719853-4c4b-4f4f-bcc3-2cc539e6da52-ovn-node-metrics-cert\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492798 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-log-socket\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492823 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdtqt\" (UniqueName: \"kubernetes.io/projected/29719853-4c4b-4f4f-bcc3-2cc539e6da52-kube-api-access-tdtqt\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492856 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljj45\" (UniqueName: \"kubernetes.io/projected/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-kube-api-access-ljj45\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492901 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-var-lib-cni-multus\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492961 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-node-log\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.492993 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-systemd-units\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.493025 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-etc-openvswitch\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.493061 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/29719853-4c4b-4f4f-bcc3-2cc539e6da52-ovnkube-config\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.493155 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-log-socket\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.497241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.493552 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/29719853-4c4b-4f4f-bcc3-2cc539e6da52-run-systemd\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.498442 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.498419 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/29719853-4c4b-4f4f-bcc3-2cc539e6da52-ovn-node-metrics-cert\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.498627 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.498579 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/96df30a0-a2c9-42b3-9e2c-733801aa84b2-agent-certs\") pod \"konnectivity-agent-q44t2\" (UID: \"96df30a0-a2c9-42b3-9e2c-733801aa84b2\") " pod="kube-system/konnectivity-agent-q44t2" Jul 22 16:11:45.499261 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:45.498978 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:11:45.499261 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:45.499007 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:11:45.499261 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:45.499022 2565 projected.go:196] Error preparing data for projected volume kube-api-access-zx72w for pod openshift-network-diagnostics/network-check-target-blvd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:11:45.499261 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:45.499097 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w podName:7d94b09b-161f-4498-b0ac-9f3c0b104444 nodeName:}" failed. No retries permitted until 2026-07-22 16:11:45.999079618 +0000 UTC m=+3.012246638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zx72w" (UniqueName: "kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w") pod "network-check-target-blvd8" (UID: "7d94b09b-161f-4498-b0ac-9f3c0b104444") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:11:45.504148 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.504125 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbb2d\" (UniqueName: \"kubernetes.io/projected/0097d54d-435a-471d-93f0-e9075cdec91c-kube-api-access-mbb2d\") pod \"iptables-alerter-z9cck\" (UID: \"0097d54d-435a-471d-93f0-e9075cdec91c\") " pod="openshift-network-operator/iptables-alerter-z9cck" Jul 22 16:11:45.510030 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.510001 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdtqt\" (UniqueName: \"kubernetes.io/projected/29719853-4c4b-4f4f-bcc3-2cc539e6da52-kube-api-access-tdtqt\") pod \"ovnkube-node-xjtvj\" (UID: \"29719853-4c4b-4f4f-bcc3-2cc539e6da52\") " pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.512475 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.512437 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 16:11:45.530489 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.530390 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" event={"ID":"f2872e03d19efdcaa9be42e12fa8b080","Type":"ContainerStarted","Data":"f7f392b9273de842c7dc0d2f4ddb12b03e8cd3a18605c42fba9348987cbe60f5"} Jul 22 16:11:45.532613 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.532034 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" event={"ID":"db69a2dc3be3a098e4e8d4e246ebb928","Type":"ContainerStarted","Data":"e42611a4e5d36ba8ffc535980949c16a854f6d140d119109e5afc3da358ccacb"} Jul 22 16:11:45.593440 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593357 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/592b0f74-1d67-4361-85ba-6fe61394a87e-os-release\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.593440 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593429 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-cnibin\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.593625 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593468 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-var-lib-cni-bin\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.593625 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593489 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn4cg\" (UniqueName: \"kubernetes.io/projected/536aeccf-be47-44e1-a65c-572c81ba240a-kube-api-access-zn4cg\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.593625 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593506 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzfjs\" (UniqueName: \"kubernetes.io/projected/592b0f74-1d67-4361-85ba-6fe61394a87e-kube-api-access-jzfjs\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.593625 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593506 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-cnibin\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.593625 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593514 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/592b0f74-1d67-4361-85ba-6fe61394a87e-os-release\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.593625 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593522 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-etc-kubernetes\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.593625 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593552 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-etc-kubernetes\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.593625 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593567 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b8dd\" (UniqueName: \"kubernetes.io/projected/9fafd1b8-f815-4a95-982f-fe79e9086034-kube-api-access-9b8dd\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.593625 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593603 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-sysctl-conf\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.594065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593632 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdnfk\" (UniqueName: \"kubernetes.io/projected/3564cf28-82d3-4856-8ee0-17c61398d176-kube-api-access-gdnfk\") pod \"node-ca-bpcxs\" (UID: \"3564cf28-82d3-4856-8ee0-17c61398d176\") " pod="openshift-image-registry/node-ca-bpcxs" Jul 22 16:11:45.594065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593690 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-run-multus-certs\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.594065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-sysctl-d\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.594065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593605 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-var-lib-cni-bin\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.594065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593773 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-run-multus-certs\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.594065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593792 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-sysctl-conf\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.594065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593814 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-device-dir\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.594065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593844 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/592b0f74-1d67-4361-85ba-6fe61394a87e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.594065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593870 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-sysctl-d\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.594065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593871 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9fafd1b8-f815-4a95-982f-fe79e9086034-cni-binary-copy\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.594065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593893 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-device-dir\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.594065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593923 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9fafd1b8-f815-4a95-982f-fe79e9086034-multus-daemon-config\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.594065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.593971 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6l5b\" (UniqueName: \"kubernetes.io/projected/3dc7600c-4705-48d3-a410-eebfaa40c9cf-kube-api-access-f6l5b\") pod \"network-metrics-daemon-gkw9w\" (UID: \"3dc7600c-4705-48d3-a410-eebfaa40c9cf\") " pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:45.594065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594007 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-sys\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.594065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594036 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-multus-cni-dir\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.594065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594069 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-sysconfig\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594095 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-kubernetes\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594125 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/592b0f74-1d67-4361-85ba-6fe61394a87e-system-cni-dir\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594151 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3564cf28-82d3-4856-8ee0-17c61398d176-host\") pod \"node-ca-bpcxs\" (UID: \"3564cf28-82d3-4856-8ee0-17c61398d176\") " pod="openshift-image-registry/node-ca-bpcxs" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594160 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-sys\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594179 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-multus-socket-dir-parent\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594212 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6aae32f3-293e-4611-b044-892e51135c16-tmp-dir\") pod \"node-resolver-st8vx\" (UID: \"6aae32f3-293e-4611-b044-892e51135c16\") " pod="openshift-dns/node-resolver-st8vx" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594225 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-kubernetes\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594247 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-os-release\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594273 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-multus-cni-dir\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594279 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/592b0f74-1d67-4361-85ba-6fe61394a87e-cnibin\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594279 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-sysconfig\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594326 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-multus-socket-dir-parent\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594347 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/592b0f74-1d67-4361-85ba-6fe61394a87e-cnibin\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594359 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-os-release\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594375 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/592b0f74-1d67-4361-85ba-6fe61394a87e-system-cni-dir\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594391 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-run\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594404 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3564cf28-82d3-4856-8ee0-17c61398d176-host\") pod \"node-ca-bpcxs\" (UID: \"3564cf28-82d3-4856-8ee0-17c61398d176\") " pod="openshift-image-registry/node-ca-bpcxs" Jul 22 16:11:45.594890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594466 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/536aeccf-be47-44e1-a65c-572c81ba240a-etc-tuned\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594500 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-socket-dir\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594527 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-registration-dir\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594531 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9fafd1b8-f815-4a95-982f-fe79e9086034-cni-binary-copy\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594539 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-run\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594547 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/592b0f74-1d67-4361-85ba-6fe61394a87e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594568 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-system-cni-dir\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594595 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-var-lib-kubelet\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594619 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-registration-dir\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594636 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs\") pod \"network-metrics-daemon-gkw9w\" (UID: \"3dc7600c-4705-48d3-a410-eebfaa40c9cf\") " pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594665 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-systemd\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594671 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-system-cni-dir\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594666 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-socket-dir\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594688 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6aae32f3-293e-4611-b044-892e51135c16-tmp-dir\") pod \"node-resolver-st8vx\" (UID: \"6aae32f3-293e-4611-b044-892e51135c16\") " pod="openshift-dns/node-resolver-st8vx" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594691 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-lib-modules\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594710 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-var-lib-kubelet\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594729 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/536aeccf-be47-44e1-a65c-572c81ba240a-tmp\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594740 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-systemd\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.595792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594769 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-hostroot\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594799 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-multus-conf-dir\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594830 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-host\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:45.594835 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594854 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-lib-modules\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594867 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-sys-fs\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:45.594897 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs podName:3dc7600c-4705-48d3-a410-eebfaa40c9cf nodeName:}" failed. No retries permitted until 2026-07-22 16:11:46.094875807 +0000 UTC m=+3.108042828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs") pod "network-metrics-daemon-gkw9w" (UID: "3dc7600c-4705-48d3-a410-eebfaa40c9cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594915 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-sys-fs\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594942 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-host\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594956 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-hostroot\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594971 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-run-k8s-cni-cncf-io\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594996 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-multus-conf-dir\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.594998 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-modprobe-d\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595028 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-var-lib-kubelet\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595053 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595080 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-etc-modprobe-d\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595088 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6aae32f3-293e-4611-b044-892e51135c16-hosts-file\") pod \"node-resolver-st8vx\" (UID: \"6aae32f3-293e-4611-b044-892e51135c16\") " pod="openshift-dns/node-resolver-st8vx" Jul 22 16:11:45.596701 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595115 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/536aeccf-be47-44e1-a65c-572c81ba240a-var-lib-kubelet\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595121 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3564cf28-82d3-4856-8ee0-17c61398d176-serviceca\") pod \"node-ca-bpcxs\" (UID: \"3564cf28-82d3-4856-8ee0-17c61398d176\") " pod="openshift-image-registry/node-ca-bpcxs" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595130 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-run-k8s-cni-cncf-io\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595149 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6aae32f3-293e-4611-b044-892e51135c16-hosts-file\") pod \"node-resolver-st8vx\" (UID: \"6aae32f3-293e-4611-b044-892e51135c16\") " pod="openshift-dns/node-resolver-st8vx" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595272 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljj45\" (UniqueName: \"kubernetes.io/projected/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-kube-api-access-ljj45\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595303 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-var-lib-cni-multus\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595350 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595379 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/592b0f74-1d67-4361-85ba-6fe61394a87e-cni-binary-copy\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595426 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/592b0f74-1d67-4361-85ba-6fe61394a87e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595459 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgh4m\" (UniqueName: \"kubernetes.io/projected/6aae32f3-293e-4611-b044-892e51135c16-kube-api-access-tgh4m\") pod \"node-resolver-st8vx\" (UID: \"6aae32f3-293e-4611-b044-892e51135c16\") " pod="openshift-dns/node-resolver-st8vx" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595491 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-run-netns\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595518 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-etc-selinux\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595522 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/592b0f74-1d67-4361-85ba-6fe61394a87e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595605 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-var-lib-cni-multus\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595621 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595757 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/592b0f74-1d67-4361-85ba-6fe61394a87e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.595959 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3564cf28-82d3-4856-8ee0-17c61398d176-serviceca\") pod \"node-ca-bpcxs\" (UID: \"3564cf28-82d3-4856-8ee0-17c61398d176\") " pod="openshift-image-registry/node-ca-bpcxs" Jul 22 16:11:45.597618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.596012 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9fafd1b8-f815-4a95-982f-fe79e9086034-host-run-netns\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.598515 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.596282 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9fafd1b8-f815-4a95-982f-fe79e9086034-multus-daemon-config\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.598515 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.596296 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/592b0f74-1d67-4361-85ba-6fe61394a87e-cni-binary-copy\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.598515 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.596631 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/592b0f74-1d67-4361-85ba-6fe61394a87e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.598941 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.598890 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/536aeccf-be47-44e1-a65c-572c81ba240a-tmp\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.603916 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.603891 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/536aeccf-be47-44e1-a65c-572c81ba240a-etc-tuned\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.614996 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.614964 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljj45\" (UniqueName: \"kubernetes.io/projected/ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8-kube-api-access-ljj45\") pod \"aws-ebs-csi-driver-node-m24ll\" (UID: \"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.617066 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.615444 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdnfk\" (UniqueName: \"kubernetes.io/projected/3564cf28-82d3-4856-8ee0-17c61398d176-kube-api-access-gdnfk\") pod \"node-ca-bpcxs\" (UID: \"3564cf28-82d3-4856-8ee0-17c61398d176\") " pod="openshift-image-registry/node-ca-bpcxs" Jul 22 16:11:45.622108 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.622081 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzfjs\" (UniqueName: \"kubernetes.io/projected/592b0f74-1d67-4361-85ba-6fe61394a87e-kube-api-access-jzfjs\") pod \"multus-additional-cni-plugins-bj4s6\" (UID: \"592b0f74-1d67-4361-85ba-6fe61394a87e\") " pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.622769 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.622747 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgh4m\" (UniqueName: \"kubernetes.io/projected/6aae32f3-293e-4611-b044-892e51135c16-kube-api-access-tgh4m\") pod \"node-resolver-st8vx\" (UID: \"6aae32f3-293e-4611-b044-892e51135c16\") " pod="openshift-dns/node-resolver-st8vx" Jul 22 16:11:45.623213 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.623193 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b8dd\" (UniqueName: \"kubernetes.io/projected/9fafd1b8-f815-4a95-982f-fe79e9086034-kube-api-access-9b8dd\") pod \"multus-vwml7\" (UID: \"9fafd1b8-f815-4a95-982f-fe79e9086034\") " pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.624698 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.624673 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6l5b\" (UniqueName: \"kubernetes.io/projected/3dc7600c-4705-48d3-a410-eebfaa40c9cf-kube-api-access-f6l5b\") pod \"network-metrics-daemon-gkw9w\" (UID: \"3dc7600c-4705-48d3-a410-eebfaa40c9cf\") " pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:45.626797 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.626777 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn4cg\" (UniqueName: \"kubernetes.io/projected/536aeccf-be47-44e1-a65c-572c81ba240a-kube-api-access-zn4cg\") pod \"tuned-6lbnb\" (UID: \"536aeccf-be47-44e1-a65c-572c81ba240a\") " pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.688427 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.688176 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z9cck" Jul 22 16:11:45.698514 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.698291 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:11:45.708094 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.708072 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-q44t2" Jul 22 16:11:45.715575 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.715555 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" Jul 22 16:11:45.725058 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.725039 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-st8vx" Jul 22 16:11:45.736195 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.736039 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bpcxs" Jul 22 16:11:45.745716 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.745695 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vwml7" Jul 22 16:11:45.753604 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.753577 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" Jul 22 16:11:45.761500 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.761467 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bj4s6" Jul 22 16:11:45.807118 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:45.806953 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod592b0f74_1d67_4361_85ba_6fe61394a87e.slice/crio-001b41cd45dcf18469646f6cca14cdc784187bead74f142a10e1e764c8ea5045 WatchSource:0}: Error finding container 001b41cd45dcf18469646f6cca14cdc784187bead74f142a10e1e764c8ea5045: Status 404 returned error can't find the container with id 001b41cd45dcf18469646f6cca14cdc784187bead74f142a10e1e764c8ea5045 Jul 22 16:11:45.809337 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:45.809296 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod536aeccf_be47_44e1_a65c_572c81ba240a.slice/crio-46f518cb03321c6a4f7db9028a0daedfbaf9d58d47d25dfea8e468117da82ffb WatchSource:0}: Error finding container 46f518cb03321c6a4f7db9028a0daedfbaf9d58d47d25dfea8e468117da82ffb: Status 404 returned error can't find the container with id 46f518cb03321c6a4f7db9028a0daedfbaf9d58d47d25dfea8e468117da82ffb Jul 22 16:11:45.817926 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:45.817901 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aae32f3_293e_4611_b044_892e51135c16.slice/crio-1f38078964f4e3f1b9735f8e229dd96ce8e608a8a37ca22e55c33251202d9105 WatchSource:0}: Error finding container 1f38078964f4e3f1b9735f8e229dd96ce8e608a8a37ca22e55c33251202d9105: Status 404 returned error can't find the container with id 1f38078964f4e3f1b9735f8e229dd96ce8e608a8a37ca22e55c33251202d9105 Jul 22 16:11:45.839844 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:45.839809 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0097d54d_435a_471d_93f0_e9075cdec91c.slice/crio-5a8bde33251597ba6a0b99b59236f3846197f6eeb7f7cc7b1ad07a500895995e WatchSource:0}: Error finding container 5a8bde33251597ba6a0b99b59236f3846197f6eeb7f7cc7b1ad07a500895995e: Status 404 returned error can't find the container with id 5a8bde33251597ba6a0b99b59236f3846197f6eeb7f7cc7b1ad07a500895995e Jul 22 16:11:45.842155 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:45.842128 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecb38e0b_1422_4aae_acc7_cfaf0b9d32c8.slice/crio-36f2fad46864a947609ac934dfb3c58a74152705d4daf74ccf10d3425ab87998 WatchSource:0}: Error finding container 36f2fad46864a947609ac934dfb3c58a74152705d4daf74ccf10d3425ab87998: Status 404 returned error can't find the container with id 36f2fad46864a947609ac934dfb3c58a74152705d4daf74ccf10d3425ab87998 Jul 22 16:11:45.842666 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:45.842470 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fafd1b8_f815_4a95_982f_fe79e9086034.slice/crio-e326da4473e29fef88453eeb9cedc33bb442671469decbfa9299b97a299c9f66 WatchSource:0}: Error finding container e326da4473e29fef88453eeb9cedc33bb442671469decbfa9299b97a299c9f66: Status 404 returned error can't find the container with id e326da4473e29fef88453eeb9cedc33bb442671469decbfa9299b97a299c9f66 Jul 22 16:11:45.842943 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:45.842910 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3564cf28_82d3_4856_8ee0_17c61398d176.slice/crio-b9f0d0c00e71c7020c7e045ba76e3721b130a03c62579aadf5485c7719b5337a WatchSource:0}: Error finding container b9f0d0c00e71c7020c7e045ba76e3721b130a03c62579aadf5485c7719b5337a: Status 404 returned error can't find the container with id b9f0d0c00e71c7020c7e045ba76e3721b130a03c62579aadf5485c7719b5337a Jul 22 16:11:45.844437 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:45.843757 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96df30a0_a2c9_42b3_9e2c_733801aa84b2.slice/crio-240077951db2d5c74775b32eef7fe66531a200632f9c1e725ab343b1b38cd166 WatchSource:0}: Error finding container 240077951db2d5c74775b32eef7fe66531a200632f9c1e725ab343b1b38cd166: Status 404 returned error can't find the container with id 240077951db2d5c74775b32eef7fe66531a200632f9c1e725ab343b1b38cd166 Jul 22 16:11:45.844437 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:11:45.844234 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29719853_4c4b_4f4f_bcc3_2cc539e6da52.slice/crio-fd1f51e3cb72fda69074c20000d3aa7a455d7384efb7092a07ed1722695327c4 WatchSource:0}: Error finding container fd1f51e3cb72fda69074c20000d3aa7a455d7384efb7092a07ed1722695327c4: Status 404 returned error can't find the container with id fd1f51e3cb72fda69074c20000d3aa7a455d7384efb7092a07ed1722695327c4 Jul 22 16:11:45.997885 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:45.997852 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret\") pod \"global-pull-secret-syncer-jdctp\" (UID: \"6b58efe5-eede-4fbf-a506-ae944342040f\") " pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:45.998054 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:45.997994 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:11:45.998136 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:45.998064 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret podName:6b58efe5-eede-4fbf-a506-ae944342040f nodeName:}" failed. No retries permitted until 2026-07-22 16:11:46.998045559 +0000 UTC m=+4.011212597 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret") pod "global-pull-secret-syncer-jdctp" (UID: "6b58efe5-eede-4fbf-a506-ae944342040f") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:11:46.099157 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:46.099126 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs\") pod \"network-metrics-daemon-gkw9w\" (UID: \"3dc7600c-4705-48d3-a410-eebfaa40c9cf\") " pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:46.099249 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:46.099197 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx72w\" (UniqueName: \"kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w\") pod \"network-check-target-blvd8\" (UID: \"7d94b09b-161f-4498-b0ac-9f3c0b104444\") " pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:11:46.099295 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:46.099278 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:11:46.099366 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:46.099346 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:11:46.099436 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:46.099370 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs podName:3dc7600c-4705-48d3-a410-eebfaa40c9cf nodeName:}" failed. No retries permitted until 2026-07-22 16:11:47.099349436 +0000 UTC m=+4.112516475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs") pod "network-metrics-daemon-gkw9w" (UID: "3dc7600c-4705-48d3-a410-eebfaa40c9cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:11:46.099436 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:46.099375 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:11:46.099436 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:46.099391 2565 projected.go:196] Error preparing data for projected volume kube-api-access-zx72w for pod openshift-network-diagnostics/network-check-target-blvd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:11:46.099559 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:46.099474 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w podName:7d94b09b-161f-4498-b0ac-9f3c0b104444 nodeName:}" failed. No retries permitted until 2026-07-22 16:11:47.09945598 +0000 UTC m=+4.112623005 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zx72w" (UniqueName: "kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w") pod "network-check-target-blvd8" (UID: "7d94b09b-161f-4498-b0ac-9f3c0b104444") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:11:46.463632 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:46.463542 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 16:06:45 +0000 UTC" deadline="2028-02-14 14:02:16.105463332 +0000 UTC" Jul 22 16:11:46.463632 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:46.463584 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13725h50m29.641884203s" Jul 22 16:11:46.523298 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:46.522735 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:46.523298 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:46.522877 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:11:46.537681 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:46.537648 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vwml7" event={"ID":"9fafd1b8-f815-4a95-982f-fe79e9086034","Type":"ContainerStarted","Data":"e326da4473e29fef88453eeb9cedc33bb442671469decbfa9299b97a299c9f66"} Jul 22 16:11:46.540486 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:46.540448 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" event={"ID":"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8","Type":"ContainerStarted","Data":"36f2fad46864a947609ac934dfb3c58a74152705d4daf74ccf10d3425ab87998"} Jul 22 16:11:46.543551 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:46.543455 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bpcxs" event={"ID":"3564cf28-82d3-4856-8ee0-17c61398d176","Type":"ContainerStarted","Data":"b9f0d0c00e71c7020c7e045ba76e3721b130a03c62579aadf5485c7719b5337a"} Jul 22 16:11:46.545245 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:46.545144 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-st8vx" event={"ID":"6aae32f3-293e-4611-b044-892e51135c16","Type":"ContainerStarted","Data":"1f38078964f4e3f1b9735f8e229dd96ce8e608a8a37ca22e55c33251202d9105"} Jul 22 16:11:46.548416 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:46.547640 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" event={"ID":"f2872e03d19efdcaa9be42e12fa8b080","Type":"ContainerStarted","Data":"1338897b602700d8e102c3dadbf2e5de634883d7b2244d519dfef5c59d657c6d"} Jul 22 16:11:46.551323 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:46.551269 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q44t2" event={"ID":"96df30a0-a2c9-42b3-9e2c-733801aa84b2","Type":"ContainerStarted","Data":"240077951db2d5c74775b32eef7fe66531a200632f9c1e725ab343b1b38cd166"} Jul 22 16:11:46.554083 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:46.554059 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z9cck" event={"ID":"0097d54d-435a-471d-93f0-e9075cdec91c","Type":"ContainerStarted","Data":"5a8bde33251597ba6a0b99b59236f3846197f6eeb7f7cc7b1ad07a500895995e"} Jul 22 16:11:46.557631 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:46.557558 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" event={"ID":"536aeccf-be47-44e1-a65c-572c81ba240a","Type":"ContainerStarted","Data":"46f518cb03321c6a4f7db9028a0daedfbaf9d58d47d25dfea8e468117da82ffb"} Jul 22 16:11:46.559876 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:46.559846 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bj4s6" event={"ID":"592b0f74-1d67-4361-85ba-6fe61394a87e","Type":"ContainerStarted","Data":"001b41cd45dcf18469646f6cca14cdc784187bead74f142a10e1e764c8ea5045"} Jul 22 16:11:46.565311 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:46.565184 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" event={"ID":"29719853-4c4b-4f4f-bcc3-2cc539e6da52","Type":"ContainerStarted","Data":"fd1f51e3cb72fda69074c20000d3aa7a455d7384efb7092a07ed1722695327c4"} Jul 22 16:11:47.007072 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:47.006437 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret\") pod \"global-pull-secret-syncer-jdctp\" (UID: \"6b58efe5-eede-4fbf-a506-ae944342040f\") " pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:47.007072 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:47.006613 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:11:47.007072 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:47.006683 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret podName:6b58efe5-eede-4fbf-a506-ae944342040f nodeName:}" failed. No retries permitted until 2026-07-22 16:11:49.00666322 +0000 UTC m=+6.019830242 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret") pod "global-pull-secret-syncer-jdctp" (UID: "6b58efe5-eede-4fbf-a506-ae944342040f") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:11:47.107495 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:47.107460 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx72w\" (UniqueName: \"kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w\") pod \"network-check-target-blvd8\" (UID: \"7d94b09b-161f-4498-b0ac-9f3c0b104444\") " pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:11:47.107627 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:47.107569 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs\") pod \"network-metrics-daemon-gkw9w\" (UID: \"3dc7600c-4705-48d3-a410-eebfaa40c9cf\") " pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:47.107715 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:47.107695 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:11:47.107779 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:47.107767 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs podName:3dc7600c-4705-48d3-a410-eebfaa40c9cf nodeName:}" failed. No retries permitted until 2026-07-22 16:11:49.107747524 +0000 UTC m=+6.120914546 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs") pod "network-metrics-daemon-gkw9w" (UID: "3dc7600c-4705-48d3-a410-eebfaa40c9cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:11:47.108203 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:47.108181 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:11:47.108276 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:47.108211 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:11:47.108276 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:47.108225 2565 projected.go:196] Error preparing data for projected volume kube-api-access-zx72w for pod openshift-network-diagnostics/network-check-target-blvd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:11:47.108276 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:47.108274 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w podName:7d94b09b-161f-4498-b0ac-9f3c0b104444 nodeName:}" failed. No retries permitted until 2026-07-22 16:11:49.108257122 +0000 UTC m=+6.121424143 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zx72w" (UniqueName: "kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w") pod "network-check-target-blvd8" (UID: "7d94b09b-161f-4498-b0ac-9f3c0b104444") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:11:47.464087 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:47.464018 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 16:06:45 +0000 UTC" deadline="2028-01-03 21:27:57.23397496 +0000 UTC" Jul 22 16:11:47.464087 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:47.464087 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12725h16m9.769892618s" Jul 22 16:11:47.524822 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:47.524047 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:47.524822 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:47.524212 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:11:47.524822 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:47.524685 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:11:47.524822 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:47.524784 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:11:47.589626 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:47.589589 2565 generic.go:356] "Generic (PLEG): container finished" podID="f2872e03d19efdcaa9be42e12fa8b080" containerID="1338897b602700d8e102c3dadbf2e5de634883d7b2244d519dfef5c59d657c6d" exitCode=0 Jul 22 16:11:47.589777 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:47.589697 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" event={"ID":"f2872e03d19efdcaa9be42e12fa8b080","Type":"ContainerDied","Data":"1338897b602700d8e102c3dadbf2e5de634883d7b2244d519dfef5c59d657c6d"} Jul 22 16:11:47.612651 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:47.612194 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" event={"ID":"db69a2dc3be3a098e4e8d4e246ebb928","Type":"ContainerStarted","Data":"a665b2c7a53fdc8b7272f4080cdb5b1f6b84d250532fba11d7eb3cc0b5a1d942"} Jul 22 16:11:48.522731 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:48.522691 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:48.523239 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:48.522863 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:11:48.622115 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:48.621706 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" event={"ID":"f2872e03d19efdcaa9be42e12fa8b080","Type":"ContainerStarted","Data":"e8d742d9e365cab4f2cb0392faa8a47bff44ee8e1dccafaa0e50eb6be0e87f87"} Jul 22 16:11:48.644871 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:48.644812 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-160.ec2.internal" podStartSLOduration=4.644790597 podStartE2EDuration="4.644790597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:11:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:11:47.651050008 +0000 UTC m=+4.664217054" watchObservedRunningTime="2026-07-22 16:11:48.644790597 +0000 UTC m=+5.657957635" Jul 22 16:11:49.027291 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:49.026642 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret\") pod \"global-pull-secret-syncer-jdctp\" (UID: \"6b58efe5-eede-4fbf-a506-ae944342040f\") " pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:49.027291 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:49.026864 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:11:49.027291 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:49.026928 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret podName:6b58efe5-eede-4fbf-a506-ae944342040f nodeName:}" failed. No retries permitted until 2026-07-22 16:11:53.02690896 +0000 UTC m=+10.040075982 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret") pod "global-pull-secret-syncer-jdctp" (UID: "6b58efe5-eede-4fbf-a506-ae944342040f") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:11:49.128905 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:49.128025 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs\") pod \"network-metrics-daemon-gkw9w\" (UID: \"3dc7600c-4705-48d3-a410-eebfaa40c9cf\") " pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:49.128905 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:49.128084 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx72w\" (UniqueName: \"kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w\") pod \"network-check-target-blvd8\" (UID: \"7d94b09b-161f-4498-b0ac-9f3c0b104444\") " pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:11:49.128905 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:49.128239 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:11:49.128905 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:49.128258 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:11:49.128905 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:49.128270 2565 projected.go:196] Error preparing data for projected volume kube-api-access-zx72w for pod openshift-network-diagnostics/network-check-target-blvd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:11:49.128905 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:49.128329 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w podName:7d94b09b-161f-4498-b0ac-9f3c0b104444 nodeName:}" failed. No retries permitted until 2026-07-22 16:11:53.128311553 +0000 UTC m=+10.141478588 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zx72w" (UniqueName: "kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w") pod "network-check-target-blvd8" (UID: "7d94b09b-161f-4498-b0ac-9f3c0b104444") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:11:49.128905 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:49.128817 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:11:49.128905 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:49.128872 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs podName:3dc7600c-4705-48d3-a410-eebfaa40c9cf nodeName:}" failed. No retries permitted until 2026-07-22 16:11:53.12885653 +0000 UTC m=+10.142023554 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs") pod "network-metrics-daemon-gkw9w" (UID: "3dc7600c-4705-48d3-a410-eebfaa40c9cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:11:49.522653 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:49.522565 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:49.522653 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:49.522621 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:11:49.523250 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:49.522731 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:11:49.523250 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:49.522847 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:11:50.522800 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:50.522757 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:50.523453 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:50.522910 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:11:51.523047 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:51.523009 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:11:51.523540 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:51.523147 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:11:51.523540 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:51.523248 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:51.523540 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:51.523389 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:11:52.522990 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:52.522911 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:52.523126 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:52.523041 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:11:53.070633 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:53.070601 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret\") pod \"global-pull-secret-syncer-jdctp\" (UID: \"6b58efe5-eede-4fbf-a506-ae944342040f\") " pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:53.070819 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:53.070765 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:11:53.070893 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:53.070822 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret podName:6b58efe5-eede-4fbf-a506-ae944342040f nodeName:}" failed. No retries permitted until 2026-07-22 16:12:01.070805161 +0000 UTC m=+18.083972195 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret") pod "global-pull-secret-syncer-jdctp" (UID: "6b58efe5-eede-4fbf-a506-ae944342040f") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:11:53.172404 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:53.171613 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs\") pod \"network-metrics-daemon-gkw9w\" (UID: \"3dc7600c-4705-48d3-a410-eebfaa40c9cf\") " pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:53.172404 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:53.171670 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx72w\" (UniqueName: \"kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w\") pod \"network-check-target-blvd8\" (UID: \"7d94b09b-161f-4498-b0ac-9f3c0b104444\") " pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:11:53.172404 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:53.171818 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:11:53.172404 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:53.171835 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:11:53.172404 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:53.171848 2565 projected.go:196] Error preparing data for projected volume kube-api-access-zx72w for pod openshift-network-diagnostics/network-check-target-blvd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:11:53.172404 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:53.171902 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w podName:7d94b09b-161f-4498-b0ac-9f3c0b104444 nodeName:}" failed. No retries permitted until 2026-07-22 16:12:01.171882561 +0000 UTC m=+18.185049604 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zx72w" (UniqueName: "kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w") pod "network-check-target-blvd8" (UID: "7d94b09b-161f-4498-b0ac-9f3c0b104444") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:11:53.172404 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:53.172302 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:11:53.172404 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:53.172353 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs podName:3dc7600c-4705-48d3-a410-eebfaa40c9cf nodeName:}" failed. No retries permitted until 2026-07-22 16:12:01.172338851 +0000 UTC m=+18.185505871 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs") pod "network-metrics-daemon-gkw9w" (UID: "3dc7600c-4705-48d3-a410-eebfaa40c9cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:11:53.526329 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:53.524751 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:53.526329 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:53.524925 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:11:53.526329 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:53.525379 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:11:53.526329 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:53.525522 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:11:54.522789 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:54.522759 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:54.522954 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:54.522869 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:11:55.526181 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:55.526150 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:11:55.526649 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:55.526261 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:11:55.526649 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:55.526312 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:55.526649 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:55.526405 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:11:56.522889 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:56.522488 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:56.522889 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:56.522602 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:11:57.524962 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:57.524939 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:11:57.525268 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:57.525034 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:11:57.525268 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:57.525105 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:57.525268 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:57.525196 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:11:58.522232 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:58.522191 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:11:58.522447 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:58.522317 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:11:59.522699 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:59.522660 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:11:59.523104 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:59.522802 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:11:59.523104 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:11:59.523070 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:11:59.523226 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:11:59.523180 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:12:00.523028 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:00.522987 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:00.523468 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:00.523131 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:12:01.137262 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:01.137216 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret\") pod \"global-pull-secret-syncer-jdctp\" (UID: \"6b58efe5-eede-4fbf-a506-ae944342040f\") " pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:01.137496 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:01.137413 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:12:01.137583 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:01.137500 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret podName:6b58efe5-eede-4fbf-a506-ae944342040f nodeName:}" failed. No retries permitted until 2026-07-22 16:12:17.137477511 +0000 UTC m=+34.150644540 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret") pod "global-pull-secret-syncer-jdctp" (UID: "6b58efe5-eede-4fbf-a506-ae944342040f") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:12:01.237945 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:01.237901 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx72w\" (UniqueName: \"kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w\") pod \"network-check-target-blvd8\" (UID: \"7d94b09b-161f-4498-b0ac-9f3c0b104444\") " pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:01.238161 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:01.237983 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs\") pod \"network-metrics-daemon-gkw9w\" (UID: \"3dc7600c-4705-48d3-a410-eebfaa40c9cf\") " pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:12:01.238161 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:01.238084 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:12:01.238161 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:01.238092 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:12:01.238161 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:01.238118 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:12:01.238161 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:01.238130 2565 projected.go:196] Error preparing data for projected volume kube-api-access-zx72w for pod openshift-network-diagnostics/network-check-target-blvd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:12:01.238161 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:01.238146 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs podName:3dc7600c-4705-48d3-a410-eebfaa40c9cf nodeName:}" failed. No retries permitted until 2026-07-22 16:12:17.238128601 +0000 UTC m=+34.251295617 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs") pod "network-metrics-daemon-gkw9w" (UID: "3dc7600c-4705-48d3-a410-eebfaa40c9cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:12:01.238412 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:01.238176 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w podName:7d94b09b-161f-4498-b0ac-9f3c0b104444 nodeName:}" failed. No retries permitted until 2026-07-22 16:12:17.238157904 +0000 UTC m=+34.251324921 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zx72w" (UniqueName: "kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w") pod "network-check-target-blvd8" (UID: "7d94b09b-161f-4498-b0ac-9f3c0b104444") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:12:01.522324 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:01.522222 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:12:01.522324 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:01.522246 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:01.522590 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:01.522401 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:12:01.522590 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:01.522496 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:12:02.523020 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:02.522988 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:02.523440 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:02.523090 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:12:03.525734 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:03.525699 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:03.526155 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:03.525812 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:12:03.526155 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:03.525867 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:12:03.526155 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:03.525966 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:12:04.522654 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:04.522614 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:04.522830 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:04.522749 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:12:05.522496 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:05.522448 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:12:05.522981 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:05.522466 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:05.522981 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:05.522591 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:12:05.522981 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:05.522630 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:12:06.522481 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.522150 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:06.522645 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:06.522597 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:12:06.659064 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.658984 2565 generic.go:356] "Generic (PLEG): container finished" podID="592b0f74-1d67-4361-85ba-6fe61394a87e" containerID="53bbdb27738bdabeafce7e412d63f49876a922fca5a3e1a5913523fb3cb582c7" exitCode=0 Jul 22 16:12:06.659227 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.659073 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bj4s6" event={"ID":"592b0f74-1d67-4361-85ba-6fe61394a87e","Type":"ContainerDied","Data":"53bbdb27738bdabeafce7e412d63f49876a922fca5a3e1a5913523fb3cb582c7"} Jul 22 16:12:06.663518 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.663093 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" event={"ID":"29719853-4c4b-4f4f-bcc3-2cc539e6da52","Type":"ContainerStarted","Data":"664ac6c1bd4378f05ee1a99791e8aa50f1c9bc0838ad6deec53b7d710b65f812"} Jul 22 16:12:06.663518 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.663220 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" event={"ID":"29719853-4c4b-4f4f-bcc3-2cc539e6da52","Type":"ContainerStarted","Data":"315e5667dfbdec16ef5c62618351679769131094c66f39ab86811c294a6ad71a"} Jul 22 16:12:06.663518 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.663281 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" event={"ID":"29719853-4c4b-4f4f-bcc3-2cc539e6da52","Type":"ContainerStarted","Data":"a49aaeec79f3aa2e9fc3e692abc09a11ff63701122ab459da3c44bb3fb5f1e90"} Jul 22 16:12:06.663518 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.663330 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" event={"ID":"29719853-4c4b-4f4f-bcc3-2cc539e6da52","Type":"ContainerStarted","Data":"22724cdb7c735de2e4bcbe9b100ea85254fca603f7bc4536781967f01b9a5f15"} Jul 22 16:12:06.663518 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.663383 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" event={"ID":"29719853-4c4b-4f4f-bcc3-2cc539e6da52","Type":"ContainerStarted","Data":"a910bb27371133506a697dcc1df1ea16fc8f82df5a51d2d600e229b35a116108"} Jul 22 16:12:06.663518 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.663455 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" event={"ID":"29719853-4c4b-4f4f-bcc3-2cc539e6da52","Type":"ContainerStarted","Data":"254e73197eb43eccfc4cbb91480f6be4246cac84b35307f6b9ba9c6fcfd72748"} Jul 22 16:12:06.664942 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.664918 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vwml7" event={"ID":"9fafd1b8-f815-4a95-982f-fe79e9086034","Type":"ContainerStarted","Data":"5f7088c8929d4b18a2de49ac288a695adbaed6e06c2e909d5899cae7c02c7206"} Jul 22 16:12:06.666660 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.666618 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" event={"ID":"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8","Type":"ContainerStarted","Data":"4baf9fa35f946b8f5cc243c97c968cf5cfcedca7dd1944e21d3f9210f487f994"} Jul 22 16:12:06.668942 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.668918 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bpcxs" event={"ID":"3564cf28-82d3-4856-8ee0-17c61398d176","Type":"ContainerStarted","Data":"7e225421339192b378e0236886707fbedacc0e3b868f455cc6c74b62af553ab1"} Jul 22 16:12:06.670513 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.670490 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-st8vx" event={"ID":"6aae32f3-293e-4611-b044-892e51135c16","Type":"ContainerStarted","Data":"5f2227a7ba14c1c17fa1538e68b5adc6cc7f94c9c213c5a7f2e4ae6ab73d750e"} Jul 22 16:12:06.672317 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.672288 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-q44t2" event={"ID":"96df30a0-a2c9-42b3-9e2c-733801aa84b2","Type":"ContainerStarted","Data":"ae585c7aa9e97d7c3101b0003f24a24800c1cf912480ef2f7d6a6006f1b2bde9"} Jul 22 16:12:06.674158 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.674135 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" event={"ID":"536aeccf-be47-44e1-a65c-572c81ba240a","Type":"ContainerStarted","Data":"e48eaabf6941b6e1c7d0d9965fc09664cc1891fc53f233d4cc7932eb55726dec"} Jul 22 16:12:06.710796 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.710745 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-160.ec2.internal" podStartSLOduration=22.710730235 podStartE2EDuration="22.710730235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:11:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:11:48.646258718 +0000 UTC m=+5.659425762" watchObservedRunningTime="2026-07-22 16:12:06.710730235 +0000 UTC m=+23.723897273" Jul 22 16:12:06.735666 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.735619 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-q44t2" podStartSLOduration=7.319472104 podStartE2EDuration="22.735602637s" totalImagesPullingTime="15.416130533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:12:06.734829273 +0000 UTC m=+23.747996304" watchObservedRunningTime="2026-07-22 16:12:06.735602637 +0000 UTC m=+23.748769676" Jul 22 16:12:06.771397 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.771348 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bpcxs" podStartSLOduration=2.82889991 podStartE2EDuration="22.771332822s" totalImagesPullingTime="19.942432912s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:12:06.77121941 +0000 UTC m=+23.784386450" watchObservedRunningTime="2026-07-22 16:12:06.771332822 +0000 UTC m=+23.784499861" Jul 22 16:12:06.795493 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.795449 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vwml7" podStartSLOduration=2.765961229 podStartE2EDuration="22.795433353s" totalImagesPullingTime="20.029472124s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:12:06.795050572 +0000 UTC m=+23.808217610" watchObservedRunningTime="2026-07-22 16:12:06.795433353 +0000 UTC m=+23.808600392" Jul 22 16:12:06.827242 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.827191 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6lbnb" podStartSLOduration=2.850609018 podStartE2EDuration="22.827170976s" totalImagesPullingTime="19.976561958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:12:06.827089458 +0000 UTC m=+23.840256498" watchObservedRunningTime="2026-07-22 16:12:06.827170976 +0000 UTC m=+23.840338014" Jul 22 16:12:06.852966 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:06.852917 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-st8vx" podStartSLOduration=2.929491935 podStartE2EDuration="22.852901503s" totalImagesPullingTime="19.923409568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:12:06.852651246 +0000 UTC m=+23.865818287" watchObservedRunningTime="2026-07-22 16:12:06.852901503 +0000 UTC m=+23.866068542" Jul 22 16:12:07.314244 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:07.314218 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 16:12:07.448263 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:07.447974 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T16:12:07.314238135Z","UUID":"56dcd2c4-585a-4d8b-a307-79e207d795a1","Handler":null,"Name":"","Endpoint":""} Jul 22 16:12:07.449765 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:07.449740 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 16:12:07.449898 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:07.449785 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 16:12:07.523095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:07.523061 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:12:07.523631 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:07.523061 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:07.523631 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:07.523229 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:12:07.523631 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:07.523274 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:12:07.678403 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:07.678340 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z9cck" event={"ID":"0097d54d-435a-471d-93f0-e9075cdec91c","Type":"ContainerStarted","Data":"316763cd06935d318cfd3699e82d33c9cff0e8aef3f6a10f39a02c56c4ad053e"} Jul 22 16:12:07.680495 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:07.680452 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" event={"ID":"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8","Type":"ContainerStarted","Data":"e5efc1222e43e27c75b0aa545733b2375cc83d986ffcc90da762b1d6f7c68eb2"} Jul 22 16:12:07.703038 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:07.702940 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z9cck" podStartSLOduration=3.752982188 podStartE2EDuration="23.702924271s" totalImagesPullingTime="19.949942083s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:12:07.702315258 +0000 UTC m=+24.715482298" watchObservedRunningTime="2026-07-22 16:12:07.702924271 +0000 UTC m=+24.716091310" Jul 22 16:12:08.329253 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:08.329218 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-q44t2" Jul 22 16:12:08.329893 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:08.329873 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-q44t2" Jul 22 16:12:08.522034 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:08.522008 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:08.522163 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:08.522141 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:12:08.685357 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:08.685318 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" event={"ID":"ecb38e0b-1422-4aae-acc7-cfaf0b9d32c8","Type":"ContainerStarted","Data":"08bc5e6ebae7f8a47c4e3cd88c7bd957ee70b8f3a27d02e06cdb09c336988a4b"} Jul 22 16:12:08.685783 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:08.685720 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-q44t2" Jul 22 16:12:08.686182 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:08.686154 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-q44t2" Jul 22 16:12:08.709546 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:08.709496 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-m24ll" podStartSLOduration=2.406093935 podStartE2EDuration="24.709480931s" totalImagesPullingTime="22.303386996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:11:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:12:08.708817124 +0000 UTC m=+25.721984164" watchObservedRunningTime="2026-07-22 16:12:08.709480931 +0000 UTC m=+25.722647981" Jul 22 16:12:09.522284 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:09.522239 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:09.522458 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:09.522242 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:12:09.522458 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:09.522431 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:12:09.522807 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:09.522780 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:12:09.691593 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:09.691552 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" event={"ID":"29719853-4c4b-4f4f-bcc3-2cc539e6da52","Type":"ContainerStarted","Data":"40873ba1c889d995d8e091a12f771ea8ab24336e9fdceee1994b18176810ed66"} Jul 22 16:12:10.522991 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:10.522772 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:10.523165 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:10.523081 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:12:11.522427 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:11.522173 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:12:11.523267 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:11.522194 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:11.523267 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:11.522482 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:12:11.523267 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:11.522539 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:12:11.697756 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:11.697717 2565 generic.go:356] "Generic (PLEG): container finished" podID="592b0f74-1d67-4361-85ba-6fe61394a87e" containerID="fb1a549a27aefbfad24fc9248ed5deda8d8d67c49274f6ef1ff51451f03c3e45" exitCode=0 Jul 22 16:12:11.697926 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:11.697775 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bj4s6" event={"ID":"592b0f74-1d67-4361-85ba-6fe61394a87e","Type":"ContainerDied","Data":"fb1a549a27aefbfad24fc9248ed5deda8d8d67c49274f6ef1ff51451f03c3e45"} Jul 22 16:12:11.701550 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:11.701521 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" event={"ID":"29719853-4c4b-4f4f-bcc3-2cc539e6da52","Type":"ContainerStarted","Data":"b13c2ff0377071d731bf7fe87974777e7e8639631f4a822469c951ebae038694"} Jul 22 16:12:11.701908 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:11.701886 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:12:11.702046 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:11.701950 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:12:11.718555 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:11.718536 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:12:11.767783 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:11.767736 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" podStartSLOduration=7.762143748 podStartE2EDuration="27.767720141s" totalImagesPullingTime="20.005576393s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:12:11.76738239 +0000 UTC m=+28.780549428" watchObservedRunningTime="2026-07-22 16:12:11.767720141 +0000 UTC m=+28.780887180" Jul 22 16:12:12.522900 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:12.522813 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:12.523374 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:12.522914 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:12:12.704367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:12.704335 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:12:12.719298 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:12.719274 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:12:13.012774 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:13.012744 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-blvd8"] Jul 22 16:12:13.012907 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:13.012888 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:13.012996 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:13.012975 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:12:13.015642 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:13.015615 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jdctp"] Jul 22 16:12:13.015751 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:13.015697 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:13.015797 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:13.015764 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:12:13.027199 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:13.027172 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gkw9w"] Jul 22 16:12:13.027309 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:13.027262 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:12:13.027373 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:13.027356 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:12:13.707740 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:13.707711 2565 generic.go:356] "Generic (PLEG): container finished" podID="592b0f74-1d67-4361-85ba-6fe61394a87e" containerID="0da77f2dc5891e854d05156ae6e30fb78208b7316f07398e3934e76ca098bd2b" exitCode=0 Jul 22 16:12:13.708137 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:13.707800 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bj4s6" event={"ID":"592b0f74-1d67-4361-85ba-6fe61394a87e","Type":"ContainerDied","Data":"0da77f2dc5891e854d05156ae6e30fb78208b7316f07398e3934e76ca098bd2b"} Jul 22 16:12:14.522073 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:14.522042 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:12:14.522217 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:14.522041 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:14.522217 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:14.522177 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:12:14.522217 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:14.522048 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:14.522378 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:14.522213 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:12:14.522378 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:14.522307 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:12:15.717031 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:15.716846 2565 generic.go:356] "Generic (PLEG): container finished" podID="592b0f74-1d67-4361-85ba-6fe61394a87e" containerID="f418e29a2ef490b378756f107095127a1b0573af6a8e7c93f518d788292ab53f" exitCode=0 Jul 22 16:12:15.717487 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:15.716927 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bj4s6" event={"ID":"592b0f74-1d67-4361-85ba-6fe61394a87e","Type":"ContainerDied","Data":"f418e29a2ef490b378756f107095127a1b0573af6a8e7c93f518d788292ab53f"} Jul 22 16:12:16.522784 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:16.522740 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:16.523015 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:16.522762 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:12:16.523015 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:16.522881 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:12:16.523015 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:16.522762 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:16.523015 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:16.522955 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:12:16.523194 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:16.523035 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:12:17.156843 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:17.156814 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret\") pod \"global-pull-secret-syncer-jdctp\" (UID: \"6b58efe5-eede-4fbf-a506-ae944342040f\") " pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:17.157242 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:17.156960 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 16:12:17.157242 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:17.157037 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret podName:6b58efe5-eede-4fbf-a506-ae944342040f nodeName:}" failed. No retries permitted until 2026-07-22 16:12:49.157017389 +0000 UTC m=+66.170184425 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret") pod "global-pull-secret-syncer-jdctp" (UID: "6b58efe5-eede-4fbf-a506-ae944342040f") : object "kube-system"/"original-pull-secret" not registered Jul 22 16:12:17.257656 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:17.257621 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs\") pod \"network-metrics-daemon-gkw9w\" (UID: \"3dc7600c-4705-48d3-a410-eebfaa40c9cf\") " pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:12:17.257830 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:17.257669 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx72w\" (UniqueName: \"kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w\") pod \"network-check-target-blvd8\" (UID: \"7d94b09b-161f-4498-b0ac-9f3c0b104444\") " pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:17.257830 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:17.257792 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 16:12:17.257830 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:17.257794 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:12:17.257830 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:17.257810 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 16:12:17.257830 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:17.257822 2565 projected.go:196] Error preparing data for projected volume kube-api-access-zx72w for pod openshift-network-diagnostics/network-check-target-blvd8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:12:17.258070 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:17.257869 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs podName:3dc7600c-4705-48d3-a410-eebfaa40c9cf nodeName:}" failed. No retries permitted until 2026-07-22 16:12:49.257848106 +0000 UTC m=+66.271015129 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs") pod "network-metrics-daemon-gkw9w" (UID: "3dc7600c-4705-48d3-a410-eebfaa40c9cf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 16:12:17.258070 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:17.257890 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w podName:7d94b09b-161f-4498-b0ac-9f3c0b104444 nodeName:}" failed. No retries permitted until 2026-07-22 16:12:49.257879479 +0000 UTC m=+66.271046512 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zx72w" (UniqueName: "kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w") pod "network-check-target-blvd8" (UID: "7d94b09b-161f-4498-b0ac-9f3c0b104444") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 16:12:18.522643 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.522601 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:18.523265 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.522652 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:12:18.523265 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.522652 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:18.523265 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:18.522748 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jdctp" podUID="6b58efe5-eede-4fbf-a506-ae944342040f" Jul 22 16:12:18.523265 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:18.522859 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gkw9w" podUID="3dc7600c-4705-48d3-a410-eebfaa40c9cf" Jul 22 16:12:18.523265 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:18.522970 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-blvd8" podUID="7d94b09b-161f-4498-b0ac-9f3c0b104444" Jul 22 16:12:18.782883 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.782849 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-160.ec2.internal" event="NodeReady" Jul 22 16:12:18.783067 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.783010 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 16:12:18.831142 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.831113 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f8ffcd8f4-666kp"] Jul 22 16:12:18.835956 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.835924 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:18.838576 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.838556 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 16:12:18.838872 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.838854 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 16:12:18.838972 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.838853 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 16:12:18.839220 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.839203 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lcgb9\"" Jul 22 16:12:18.845373 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.845354 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 16:12:18.848776 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.848752 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f8ffcd8f4-666kp"] Jul 22 16:12:18.856657 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.856635 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-m9wx5"] Jul 22 16:12:18.860177 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.860159 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-57gnc"] Jul 22 16:12:18.860323 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.860294 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:18.863121 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.863102 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 16:12:18.863293 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.863271 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-57gnc" Jul 22 16:12:18.863404 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.863346 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bj2bw\"" Jul 22 16:12:18.863585 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.863565 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 16:12:18.865787 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.865766 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 16:12:18.865905 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.865891 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 16:12:18.866160 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.866143 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-hg4lc\"" Jul 22 16:12:18.866375 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.866358 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 16:12:18.877019 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.876334 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m9wx5"] Jul 22 16:12:18.877296 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.877274 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-57gnc"] Jul 22 16:12:18.970676 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.970645 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:18.970676 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.970674 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-tmp-dir\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:18.970887 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.970705 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxh86\" (UniqueName: \"kubernetes.io/projected/790a8029-54c9-4de7-b572-10c17aae7e42-kube-api-access-rxh86\") pod \"ingress-canary-57gnc\" (UID: \"790a8029-54c9-4de7-b572-10c17aae7e42\") " pod="openshift-ingress-canary/ingress-canary-57gnc" Jul 22 16:12:18.970887 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.970791 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:18.970887 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.970818 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/26e2f0f2-77f1-45d0-88b5-bb10285392d0-installation-pull-secrets\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:18.970887 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.970852 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/26e2f0f2-77f1-45d0-88b5-bb10285392d0-image-registry-private-configuration\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:18.971049 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.970887 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-bound-sa-token\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:18.971049 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.970911 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8grf4\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-kube-api-access-8grf4\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:18.971049 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.970954 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-config-volume\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:18.971049 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.970973 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-certificates\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:18.971232 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.971084 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42kzx\" (UniqueName: \"kubernetes.io/projected/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-kube-api-access-42kzx\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:18.971232 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.971122 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/790a8029-54c9-4de7-b572-10c17aae7e42-cert\") pod \"ingress-canary-57gnc\" (UID: \"790a8029-54c9-4de7-b572-10c17aae7e42\") " pod="openshift-ingress-canary/ingress-canary-57gnc" Jul 22 16:12:18.971232 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.971159 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/26e2f0f2-77f1-45d0-88b5-bb10285392d0-ca-trust-extracted\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:18.971232 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:18.971188 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/26e2f0f2-77f1-45d0-88b5-bb10285392d0-trusted-ca\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.067525 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.067437 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q44t2_96df30a0-a2c9-42b3-9e2c-733801aa84b2/konnectivity-agent/0.log" limit=1024 Jul 22 16:12:19.072429 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.072384 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxh86\" (UniqueName: \"kubernetes.io/projected/790a8029-54c9-4de7-b572-10c17aae7e42-kube-api-access-rxh86\") pod \"ingress-canary-57gnc\" (UID: \"790a8029-54c9-4de7-b572-10c17aae7e42\") " pod="openshift-ingress-canary/ingress-canary-57gnc" Jul 22 16:12:19.072563 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.072463 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.072563 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.072484 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/26e2f0f2-77f1-45d0-88b5-bb10285392d0-installation-pull-secrets\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.072563 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.072507 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/26e2f0f2-77f1-45d0-88b5-bb10285392d0-image-registry-private-configuration\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.072563 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.072526 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-bound-sa-token\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.072563 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.072541 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8grf4\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-kube-api-access-8grf4\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.072796 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.072571 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-config-volume\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:19.072796 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:19.072616 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:12:19.072796 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:19.072636 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f8ffcd8f4-666kp: secret "image-registry-tls" not found Jul 22 16:12:19.072796 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:19.072698 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls podName:26e2f0f2-77f1-45d0-88b5-bb10285392d0 nodeName:}" failed. No retries permitted until 2026-07-22 16:12:19.572676769 +0000 UTC m=+36.585843788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls") pod "image-registry-5f8ffcd8f4-666kp" (UID: "26e2f0f2-77f1-45d0-88b5-bb10285392d0") : secret "image-registry-tls" not found Jul 22 16:12:19.072973 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.072902 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-certificates\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.073018 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.072970 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42kzx\" (UniqueName: \"kubernetes.io/projected/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-kube-api-access-42kzx\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:19.073018 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.072999 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/790a8029-54c9-4de7-b572-10c17aae7e42-cert\") pod \"ingress-canary-57gnc\" (UID: \"790a8029-54c9-4de7-b572-10c17aae7e42\") " pod="openshift-ingress-canary/ingress-canary-57gnc" Jul 22 16:12:19.073102 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.073022 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/26e2f0f2-77f1-45d0-88b5-bb10285392d0-ca-trust-extracted\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.073102 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.073048 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/26e2f0f2-77f1-45d0-88b5-bb10285392d0-trusted-ca\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.073102 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.073081 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:19.073236 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.073104 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-tmp-dir\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:19.073236 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.073113 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-config-volume\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:19.073236 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:19.073206 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:12:19.073370 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:19.073265 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls podName:4c228a9d-7811-459b-8b7a-6dd6e6f5a76a nodeName:}" failed. No retries permitted until 2026-07-22 16:12:19.573249218 +0000 UTC m=+36.586416249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls") pod "dns-default-m9wx5" (UID: "4c228a9d-7811-459b-8b7a-6dd6e6f5a76a") : secret "dns-default-metrics-tls" not found Jul 22 16:12:19.073501 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.073477 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/26e2f0f2-77f1-45d0-88b5-bb10285392d0-ca-trust-extracted\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.073574 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.073483 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-certificates\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.073574 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.073533 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-tmp-dir\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:19.074188 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.074131 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/26e2f0f2-77f1-45d0-88b5-bb10285392d0-trusted-ca\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.077689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.077665 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/26e2f0f2-77f1-45d0-88b5-bb10285392d0-installation-pull-secrets\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.077689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.077682 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/26e2f0f2-77f1-45d0-88b5-bb10285392d0-image-registry-private-configuration\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.077848 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.077752 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/790a8029-54c9-4de7-b572-10c17aae7e42-cert\") pod \"ingress-canary-57gnc\" (UID: \"790a8029-54c9-4de7-b572-10c17aae7e42\") " pod="openshift-ingress-canary/ingress-canary-57gnc" Jul 22 16:12:19.082875 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.082829 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8grf4\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-kube-api-access-8grf4\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.084446 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.084382 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxh86\" (UniqueName: \"kubernetes.io/projected/790a8029-54c9-4de7-b572-10c17aae7e42-kube-api-access-rxh86\") pod \"ingress-canary-57gnc\" (UID: \"790a8029-54c9-4de7-b572-10c17aae7e42\") " pod="openshift-ingress-canary/ingress-canary-57gnc" Jul 22 16:12:19.085040 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.085019 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42kzx\" (UniqueName: \"kubernetes.io/projected/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-kube-api-access-42kzx\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:19.085237 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.085221 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-bound-sa-token\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.167823 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.167794 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q44t2_96df30a0-a2c9-42b3-9e2c-733801aa84b2/konnectivity-agent/0.log" limit=1024 Jul 22 16:12:19.179876 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.179844 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-57gnc" Jul 22 16:12:19.516406 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.516357 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q44t2_96df30a0-a2c9-42b3-9e2c-733801aa84b2/konnectivity-agent/0.log" limit=1024 Jul 22 16:12:19.577029 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.576993 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:19.577783 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.577056 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:19.577783 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:19.577171 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:12:19.577783 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:19.577192 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:12:19.577783 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:19.577203 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f8ffcd8f4-666kp: secret "image-registry-tls" not found Jul 22 16:12:19.577783 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:19.577242 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls podName:4c228a9d-7811-459b-8b7a-6dd6e6f5a76a nodeName:}" failed. No retries permitted until 2026-07-22 16:12:20.577220899 +0000 UTC m=+37.590387938 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls") pod "dns-default-m9wx5" (UID: "4c228a9d-7811-459b-8b7a-6dd6e6f5a76a") : secret "dns-default-metrics-tls" not found Jul 22 16:12:19.577783 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:19.577259 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls podName:26e2f0f2-77f1-45d0-88b5-bb10285392d0 nodeName:}" failed. No retries permitted until 2026-07-22 16:12:20.577250726 +0000 UTC m=+37.590417742 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls") pod "image-registry-5f8ffcd8f4-666kp" (UID: "26e2f0f2-77f1-45d0-88b5-bb10285392d0") : secret "image-registry-tls" not found Jul 22 16:12:19.613541 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.613506 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q44t2_96df30a0-a2c9-42b3-9e2c-733801aa84b2/konnectivity-agent/0.log" limit=1024 Jul 22 16:12:19.871778 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:19.871638 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-57gnc"] Jul 22 16:12:19.875371 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:12:19.875347 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod790a8029_54c9_4de7_b572_10c17aae7e42.slice/crio-bd7c4b1ee4eaf41acbb73ac560ba34272a601434bfbe6e1c325b95151713fc12 WatchSource:0}: Error finding container bd7c4b1ee4eaf41acbb73ac560ba34272a601434bfbe6e1c325b95151713fc12: Status 404 returned error can't find the container with id bd7c4b1ee4eaf41acbb73ac560ba34272a601434bfbe6e1c325b95151713fc12 Jul 22 16:12:20.522945 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:20.522914 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:12:20.523115 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:20.522914 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:20.523350 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:20.522925 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:20.526448 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:20.526429 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 16:12:20.527671 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:20.527613 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 16:12:20.528114 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:20.527934 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 16:12:20.528114 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:20.528069 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wvt7k\"" Jul 22 16:12:20.528114 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:20.528093 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 16:12:20.528114 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:20.528104 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-df68t\"" Jul 22 16:12:20.586971 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:20.586940 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:20.587360 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:20.587002 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:20.587360 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:20.587107 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:12:20.587360 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:20.587130 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:12:20.587360 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:20.587144 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f8ffcd8f4-666kp: secret "image-registry-tls" not found Jul 22 16:12:20.587360 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:20.587194 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls podName:4c228a9d-7811-459b-8b7a-6dd6e6f5a76a nodeName:}" failed. No retries permitted until 2026-07-22 16:12:22.587171611 +0000 UTC m=+39.600338630 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls") pod "dns-default-m9wx5" (UID: "4c228a9d-7811-459b-8b7a-6dd6e6f5a76a") : secret "dns-default-metrics-tls" not found Jul 22 16:12:20.587360 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:20.587215 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls podName:26e2f0f2-77f1-45d0-88b5-bb10285392d0 nodeName:}" failed. No retries permitted until 2026-07-22 16:12:22.587204835 +0000 UTC m=+39.600371859 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls") pod "image-registry-5f8ffcd8f4-666kp" (UID: "26e2f0f2-77f1-45d0-88b5-bb10285392d0") : secret "image-registry-tls" not found Jul 22 16:12:20.733634 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:20.733581 2565 generic.go:356] "Generic (PLEG): container finished" podID="592b0f74-1d67-4361-85ba-6fe61394a87e" containerID="02cc783735ad3394cfa831852dc2f7331608e5d2598991cd22fc08ad5b8b2875" exitCode=0 Jul 22 16:12:20.733817 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:20.733677 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bj4s6" event={"ID":"592b0f74-1d67-4361-85ba-6fe61394a87e","Type":"ContainerDied","Data":"02cc783735ad3394cfa831852dc2f7331608e5d2598991cd22fc08ad5b8b2875"} Jul 22 16:12:20.735053 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:20.734891 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-57gnc" event={"ID":"790a8029-54c9-4de7-b572-10c17aae7e42","Type":"ContainerStarted","Data":"bd7c4b1ee4eaf41acbb73ac560ba34272a601434bfbe6e1c325b95151713fc12"} Jul 22 16:12:21.741586 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:21.741549 2565 generic.go:356] "Generic (PLEG): container finished" podID="592b0f74-1d67-4361-85ba-6fe61394a87e" containerID="529864f9a8f02e26a0096700274e3f38449e882bb3736986822bb9f471e49f28" exitCode=0 Jul 22 16:12:21.742044 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:21.741610 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bj4s6" event={"ID":"592b0f74-1d67-4361-85ba-6fe61394a87e","Type":"ContainerDied","Data":"529864f9a8f02e26a0096700274e3f38449e882bb3736986822bb9f471e49f28"} Jul 22 16:12:22.602632 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:22.602448 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:22.602805 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:22.602653 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:22.602805 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:22.602610 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:12:22.602805 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:22.602760 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls podName:4c228a9d-7811-459b-8b7a-6dd6e6f5a76a nodeName:}" failed. No retries permitted until 2026-07-22 16:12:26.602737761 +0000 UTC m=+43.615904795 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls") pod "dns-default-m9wx5" (UID: "4c228a9d-7811-459b-8b7a-6dd6e6f5a76a") : secret "dns-default-metrics-tls" not found Jul 22 16:12:22.602805 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:22.602766 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:12:22.602805 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:22.602781 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f8ffcd8f4-666kp: secret "image-registry-tls" not found Jul 22 16:12:22.602995 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:22.602830 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls podName:26e2f0f2-77f1-45d0-88b5-bb10285392d0 nodeName:}" failed. No retries permitted until 2026-07-22 16:12:26.602815742 +0000 UTC m=+43.615982758 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls") pod "image-registry-5f8ffcd8f4-666kp" (UID: "26e2f0f2-77f1-45d0-88b5-bb10285392d0") : secret "image-registry-tls" not found Jul 22 16:12:22.746641 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:22.746559 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bj4s6" event={"ID":"592b0f74-1d67-4361-85ba-6fe61394a87e","Type":"ContainerStarted","Data":"7f86f2705a974fc855c02ae1fdbc010941cdf59a23772f298ae373cefa8ce36d"} Jul 22 16:12:22.747980 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:22.747958 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-57gnc" event={"ID":"790a8029-54c9-4de7-b572-10c17aae7e42","Type":"ContainerStarted","Data":"f683f7af24485806109c948de890c327e315e79687f050e19d169dbf68098524"} Jul 22 16:12:22.778944 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:22.778899 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bj4s6" podStartSLOduration=8.45148282 podStartE2EDuration="38.778887222s" totalImagesPullingTime="30.327404402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:11:44 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:12:22.777677811 +0000 UTC m=+39.790844850" watchObservedRunningTime="2026-07-22 16:12:22.778887222 +0000 UTC m=+39.792054255" Jul 22 16:12:22.797429 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:22.797367 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-57gnc" podStartSLOduration=2.223974452 podStartE2EDuration="4.797353056s" totalImagesPullingTime="2.573378604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:12:22.796573299 +0000 UTC m=+39.809740338" watchObservedRunningTime="2026-07-22 16:12:22.797353056 +0000 UTC m=+39.810520094" Jul 22 16:12:22.923982 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:22.923946 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q44t2_96df30a0-a2c9-42b3-9e2c-733801aa84b2/konnectivity-agent/0.log" limit=1024 Jul 22 16:12:23.024700 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:23.024626 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q44t2_96df30a0-a2c9-42b3-9e2c-733801aa84b2/konnectivity-agent/0.log" limit=1024 Jul 22 16:12:23.412726 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:23.412696 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q44t2_96df30a0-a2c9-42b3-9e2c-733801aa84b2/konnectivity-agent/0.log" limit=1024 Jul 22 16:12:23.490223 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:23.490196 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q44t2_96df30a0-a2c9-42b3-9e2c-733801aa84b2/konnectivity-agent/0.log" limit=1024 Jul 22 16:12:23.725331 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:23.725257 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q44t2_96df30a0-a2c9-42b3-9e2c-733801aa84b2/konnectivity-agent/0.log" limit=1024 Jul 22 16:12:23.825268 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:23.825207 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-q44t2_96df30a0-a2c9-42b3-9e2c-733801aa84b2/konnectivity-agent/0.log" limit=1024 Jul 22 16:12:26.633655 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:26.633614 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:26.634002 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:26.633668 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:26.634002 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:26.633758 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:12:26.634002 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:26.633763 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:12:26.634002 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:26.633771 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f8ffcd8f4-666kp: secret "image-registry-tls" not found Jul 22 16:12:26.634002 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:26.633830 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls podName:26e2f0f2-77f1-45d0-88b5-bb10285392d0 nodeName:}" failed. No retries permitted until 2026-07-22 16:12:34.633814347 +0000 UTC m=+51.646981367 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls") pod "image-registry-5f8ffcd8f4-666kp" (UID: "26e2f0f2-77f1-45d0-88b5-bb10285392d0") : secret "image-registry-tls" not found Jul 22 16:12:26.634002 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:26.633843 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls podName:4c228a9d-7811-459b-8b7a-6dd6e6f5a76a nodeName:}" failed. No retries permitted until 2026-07-22 16:12:34.633837232 +0000 UTC m=+51.647004248 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls") pod "dns-default-m9wx5" (UID: "4c228a9d-7811-459b-8b7a-6dd6e6f5a76a") : secret "dns-default-metrics-tls" not found Jul 22 16:12:34.688038 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:34.687996 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:34.688486 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:34.688096 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:34.688486 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:34.688138 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:12:34.688486 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:34.688157 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f8ffcd8f4-666kp: secret "image-registry-tls" not found Jul 22 16:12:34.688486 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:34.688209 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls podName:26e2f0f2-77f1-45d0-88b5-bb10285392d0 nodeName:}" failed. No retries permitted until 2026-07-22 16:12:50.688194255 +0000 UTC m=+67.701361272 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls") pod "image-registry-5f8ffcd8f4-666kp" (UID: "26e2f0f2-77f1-45d0-88b5-bb10285392d0") : secret "image-registry-tls" not found Jul 22 16:12:34.688486 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:34.688208 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:12:34.688486 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:34.688253 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls podName:4c228a9d-7811-459b-8b7a-6dd6e6f5a76a nodeName:}" failed. No retries permitted until 2026-07-22 16:12:50.688240335 +0000 UTC m=+67.701407352 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls") pod "dns-default-m9wx5" (UID: "4c228a9d-7811-459b-8b7a-6dd6e6f5a76a") : secret "dns-default-metrics-tls" not found Jul 22 16:12:44.734641 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:44.734604 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xjtvj" Jul 22 16:12:49.191119 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.191077 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret\") pod \"global-pull-secret-syncer-jdctp\" (UID: \"6b58efe5-eede-4fbf-a506-ae944342040f\") " pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:49.194402 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.194366 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 16:12:49.205983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.205952 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b58efe5-eede-4fbf-a506-ae944342040f-original-pull-secret\") pod \"global-pull-secret-syncer-jdctp\" (UID: \"6b58efe5-eede-4fbf-a506-ae944342040f\") " pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:49.292162 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.292130 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs\") pod \"network-metrics-daemon-gkw9w\" (UID: \"3dc7600c-4705-48d3-a410-eebfaa40c9cf\") " pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:12:49.292328 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.292184 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx72w\" (UniqueName: \"kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w\") pod \"network-check-target-blvd8\" (UID: \"7d94b09b-161f-4498-b0ac-9f3c0b104444\") " pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:49.295155 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.295138 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 16:12:49.295296 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.295280 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 16:12:49.302717 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:49.302699 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 22 16:12:49.302778 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:49.302767 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs podName:3dc7600c-4705-48d3-a410-eebfaa40c9cf nodeName:}" failed. No retries permitted until 2026-07-22 16:13:53.302748942 +0000 UTC m=+130.315915962 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs") pod "network-metrics-daemon-gkw9w" (UID: "3dc7600c-4705-48d3-a410-eebfaa40c9cf") : secret "metrics-daemon-secret" not found Jul 22 16:12:49.305204 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.305191 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 16:12:49.315148 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.315127 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx72w\" (UniqueName: \"kubernetes.io/projected/7d94b09b-161f-4498-b0ac-9f3c0b104444-kube-api-access-zx72w\") pod \"network-check-target-blvd8\" (UID: \"7d94b09b-161f-4498-b0ac-9f3c0b104444\") " pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:49.343184 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.343157 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jdctp" Jul 22 16:12:49.352461 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.352441 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wvt7k\"" Jul 22 16:12:49.360290 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.360271 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:49.484654 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.484617 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jdctp"] Jul 22 16:12:49.488463 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:12:49.488436 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b58efe5_eede_4fbf_a506_ae944342040f.slice/crio-aceecfee93371b1b9fdc3d10311ec107426cc9ccddc6b1b8ac51366cc0bf85f2 WatchSource:0}: Error finding container aceecfee93371b1b9fdc3d10311ec107426cc9ccddc6b1b8ac51366cc0bf85f2: Status 404 returned error can't find the container with id aceecfee93371b1b9fdc3d10311ec107426cc9ccddc6b1b8ac51366cc0bf85f2 Jul 22 16:12:49.500262 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.500227 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-blvd8"] Jul 22 16:12:49.504375 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:12:49.504353 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d94b09b_161f_4498_b0ac_9f3c0b104444.slice/crio-127d258fe315afe31ed2baae39c1fb22c68fed79aedcd5283c1d7a2c59e6ed0f WatchSource:0}: Error finding container 127d258fe315afe31ed2baae39c1fb22c68fed79aedcd5283c1d7a2c59e6ed0f: Status 404 returned error can't find the container with id 127d258fe315afe31ed2baae39c1fb22c68fed79aedcd5283c1d7a2c59e6ed0f Jul 22 16:12:49.811418 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.811302 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jdctp" event={"ID":"6b58efe5-eede-4fbf-a506-ae944342040f","Type":"ContainerStarted","Data":"aceecfee93371b1b9fdc3d10311ec107426cc9ccddc6b1b8ac51366cc0bf85f2"} Jul 22 16:12:49.812293 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.812272 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-blvd8" event={"ID":"7d94b09b-161f-4498-b0ac-9f3c0b104444","Type":"ContainerStarted","Data":"127d258fe315afe31ed2baae39c1fb22c68fed79aedcd5283c1d7a2c59e6ed0f"} Jul 22 16:12:49.974951 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.974917 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mgbrd"] Jul 22 16:12:49.977289 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.977260 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:49.983080 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.982877 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 16:12:49.984321 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.984296 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-88nqz\"" Jul 22 16:12:49.984637 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.984615 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 16:12:49.986485 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.986285 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 16:12:49.987280 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.987258 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 16:12:49.994953 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.994886 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mgbrd"] Jul 22 16:12:49.999267 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.999218 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-crio-socket\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:49.999425 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.999310 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhg8h\" (UniqueName: \"kubernetes.io/projected/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-api-access-fhg8h\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:49.999425 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.999349 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:49.999425 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.999415 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:49.999590 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:49.999478 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-data-volume\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:50.100689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:50.100598 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:50.100689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:50.100672 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-data-volume\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:50.100875 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:50.100699 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-crio-socket\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:50.100875 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:50.100749 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhg8h\" (UniqueName: \"kubernetes.io/projected/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-api-access-fhg8h\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:50.100875 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:50.100771 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:12:50.100875 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:50.100780 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:50.100875 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:50.100837 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls podName:4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c nodeName:}" failed. No retries permitted until 2026-07-22 16:12:50.600817801 +0000 UTC m=+67.613984821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mgbrd" (UID: "4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c") : secret "insights-runtime-extractor-tls" not found Jul 22 16:12:50.100875 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:50.100846 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 16:12:50.101106 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:50.100886 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm podName:4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c nodeName:}" failed. No retries permitted until 2026-07-22 16:12:50.600876342 +0000 UTC m=+67.614043377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mgbrd" (UID: "4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c") : configmap "kube-rbac-proxy" not found Jul 22 16:12:50.101106 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:50.100937 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-crio-socket\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:50.101106 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:50.101082 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-data-volume\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:50.113931 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:50.113897 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhg8h\" (UniqueName: \"kubernetes.io/projected/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-api-access-fhg8h\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:50.604955 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:50.604907 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:50.605414 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:50.604988 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:50.605414 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:50.605058 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 16:12:50.605414 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:50.605109 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:12:50.605414 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:50.605175 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls podName:4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c nodeName:}" failed. No retries permitted until 2026-07-22 16:12:51.605156585 +0000 UTC m=+68.618323605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mgbrd" (UID: "4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c") : secret "insights-runtime-extractor-tls" not found Jul 22 16:12:50.605414 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:50.605196 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm podName:4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c nodeName:}" failed. No retries permitted until 2026-07-22 16:12:51.605184279 +0000 UTC m=+68.618351333 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mgbrd" (UID: "4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c") : configmap "kube-rbac-proxy" not found Jul 22 16:12:50.705706 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:50.705669 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:12:50.705898 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:50.705740 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:12:50.705898 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:50.705839 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 16:12:50.705898 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:50.705879 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 16:12:50.705898 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:50.705896 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f8ffcd8f4-666kp: secret "image-registry-tls" not found Jul 22 16:12:50.706095 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:50.705922 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls podName:4c228a9d-7811-459b-8b7a-6dd6e6f5a76a nodeName:}" failed. No retries permitted until 2026-07-22 16:13:22.705901311 +0000 UTC m=+99.719068330 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls") pod "dns-default-m9wx5" (UID: "4c228a9d-7811-459b-8b7a-6dd6e6f5a76a") : secret "dns-default-metrics-tls" not found Jul 22 16:12:50.706095 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:50.705944 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls podName:26e2f0f2-77f1-45d0-88b5-bb10285392d0 nodeName:}" failed. No retries permitted until 2026-07-22 16:13:22.70593067 +0000 UTC m=+99.719097708 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls") pod "image-registry-5f8ffcd8f4-666kp" (UID: "26e2f0f2-77f1-45d0-88b5-bb10285392d0") : secret "image-registry-tls" not found Jul 22 16:12:51.612815 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:51.612775 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:51.613254 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:51.612837 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:51.613254 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:51.612924 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 16:12:51.613254 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:51.612966 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:12:51.613254 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:51.613010 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm podName:4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c nodeName:}" failed. No retries permitted until 2026-07-22 16:12:53.612987399 +0000 UTC m=+70.626154418 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mgbrd" (UID: "4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c") : configmap "kube-rbac-proxy" not found Jul 22 16:12:51.613254 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:51.613030 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls podName:4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c nodeName:}" failed. No retries permitted until 2026-07-22 16:12:53.613020868 +0000 UTC m=+70.626187945 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mgbrd" (UID: "4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c") : secret "insights-runtime-extractor-tls" not found Jul 22 16:12:51.983532 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:51.983446 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-st8vx_6aae32f3-293e-4611-b044-892e51135c16/dns-node-resolver/0.log" Jul 22 16:12:52.783937 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:52.783902 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bpcxs_3564cf28-82d3-4856-8ee0-17c61398d176/node-ca/0.log" Jul 22 16:12:53.388102 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:53.388077 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-57gnc_790a8029-54c9-4de7-b572-10c17aae7e42/serve-healthcheck-canary/0.log" Jul 22 16:12:53.627733 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:53.627696 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:53.627912 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:53.627759 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:53.627912 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:53.627800 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 16:12:53.627912 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:53.627877 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:12:53.627912 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:53.627893 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm podName:4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c nodeName:}" failed. No retries permitted until 2026-07-22 16:12:57.627871634 +0000 UTC m=+74.641038651 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mgbrd" (UID: "4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c") : configmap "kube-rbac-proxy" not found Jul 22 16:12:53.628114 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:53.627935 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls podName:4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c nodeName:}" failed. No retries permitted until 2026-07-22 16:12:57.627919828 +0000 UTC m=+74.641086890 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mgbrd" (UID: "4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c") : secret "insights-runtime-extractor-tls" not found Jul 22 16:12:54.826369 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:54.826335 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-blvd8" event={"ID":"7d94b09b-161f-4498-b0ac-9f3c0b104444","Type":"ContainerStarted","Data":"9bbf79ac7db09cb4c344baf1819ce1b1bfddd434754bdfb451cd7cad60e0658c"} Jul 22 16:12:54.826837 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:54.826454 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:12:54.827786 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:54.827768 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jdctp" event={"ID":"6b58efe5-eede-4fbf-a506-ae944342040f","Type":"ContainerStarted","Data":"fb522fab8ca680f15fdbb60081679afa13940bb6e9b608e6ebf860cc8c854ece"} Jul 22 16:12:54.852565 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:54.852519 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-blvd8" podStartSLOduration=66.217520278 podStartE2EDuration="1m10.852505706s" totalImagesPullingTime="4.634985428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:12:54.85178286 +0000 UTC m=+71.864949898" watchObservedRunningTime="2026-07-22 16:12:54.852505706 +0000 UTC m=+71.865672745" Jul 22 16:12:54.883282 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:54.883232 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jdctp" podStartSLOduration=66.227168499 podStartE2EDuration="1m10.883220741s" totalImagesPullingTime="4.656052242s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:12:54.882658418 +0000 UTC m=+71.895825456" watchObservedRunningTime="2026-07-22 16:12:54.883220741 +0000 UTC m=+71.896387780" Jul 22 16:12:55.566634 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.566603 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6"] Jul 22 16:12:55.569249 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.569234 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6" Jul 22 16:12:55.572240 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.572221 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 22 16:12:55.573431 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.573408 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-5p4nn\"" Jul 22 16:12:55.573431 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.573420 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:12:55.573577 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.573414 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 22 16:12:55.573577 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.573412 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 22 16:12:55.584523 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.584498 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6"] Jul 22 16:12:55.643155 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.643117 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5nr4\" (UniqueName: \"kubernetes.io/projected/02ac6317-0f6c-4fc4-81df-699216b0f136-kube-api-access-v5nr4\") pod \"service-ca-operator-5bcbc94c78-6ckq6\" (UID: \"02ac6317-0f6c-4fc4-81df-699216b0f136\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6" Jul 22 16:12:55.643155 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.643155 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/02ac6317-0f6c-4fc4-81df-699216b0f136-config\") pod \"service-ca-operator-5bcbc94c78-6ckq6\" (UID: \"02ac6317-0f6c-4fc4-81df-699216b0f136\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6" Jul 22 16:12:55.643350 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.643294 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/02ac6317-0f6c-4fc4-81df-699216b0f136-serving-cert\") pod \"service-ca-operator-5bcbc94c78-6ckq6\" (UID: \"02ac6317-0f6c-4fc4-81df-699216b0f136\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6" Jul 22 16:12:55.666968 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.666942 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f"] Jul 22 16:12:55.669793 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.669778 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f" Jul 22 16:12:55.672798 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.672771 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 22 16:12:55.672920 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.672771 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 22 16:12:55.672920 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.672833 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 22 16:12:55.672920 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.672865 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-gcrt8\"" Jul 22 16:12:55.672920 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.672899 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 22 16:12:55.683870 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.683849 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f"] Jul 22 16:12:55.744074 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.744039 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/592cc5de-ba98-448f-9120-4690a117a4ca-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-nzb4f\" (UID: \"592cc5de-ba98-448f-9120-4690a117a4ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f" Jul 22 16:12:55.744074 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.744072 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt72m\" (UniqueName: \"kubernetes.io/projected/592cc5de-ba98-448f-9120-4690a117a4ca-kube-api-access-wt72m\") pod \"kube-storage-version-migrator-operator-7658d9b749-nzb4f\" (UID: \"592cc5de-ba98-448f-9120-4690a117a4ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f" Jul 22 16:12:55.744309 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.744134 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/02ac6317-0f6c-4fc4-81df-699216b0f136-serving-cert\") pod \"service-ca-operator-5bcbc94c78-6ckq6\" (UID: \"02ac6317-0f6c-4fc4-81df-699216b0f136\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6" Jul 22 16:12:55.744309 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.744170 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/592cc5de-ba98-448f-9120-4690a117a4ca-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-nzb4f\" (UID: \"592cc5de-ba98-448f-9120-4690a117a4ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f" Jul 22 16:12:55.744309 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.744210 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5nr4\" (UniqueName: \"kubernetes.io/projected/02ac6317-0f6c-4fc4-81df-699216b0f136-kube-api-access-v5nr4\") pod \"service-ca-operator-5bcbc94c78-6ckq6\" (UID: \"02ac6317-0f6c-4fc4-81df-699216b0f136\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6" Jul 22 16:12:55.744309 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.744235 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/02ac6317-0f6c-4fc4-81df-699216b0f136-config\") pod \"service-ca-operator-5bcbc94c78-6ckq6\" (UID: \"02ac6317-0f6c-4fc4-81df-699216b0f136\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6" Jul 22 16:12:55.744950 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.744862 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/02ac6317-0f6c-4fc4-81df-699216b0f136-config\") pod \"service-ca-operator-5bcbc94c78-6ckq6\" (UID: \"02ac6317-0f6c-4fc4-81df-699216b0f136\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6" Jul 22 16:12:55.746508 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.746487 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/02ac6317-0f6c-4fc4-81df-699216b0f136-serving-cert\") pod \"service-ca-operator-5bcbc94c78-6ckq6\" (UID: \"02ac6317-0f6c-4fc4-81df-699216b0f136\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6" Jul 22 16:12:55.763584 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.760286 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5nr4\" (UniqueName: \"kubernetes.io/projected/02ac6317-0f6c-4fc4-81df-699216b0f136-kube-api-access-v5nr4\") pod \"service-ca-operator-5bcbc94c78-6ckq6\" (UID: \"02ac6317-0f6c-4fc4-81df-699216b0f136\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6" Jul 22 16:12:55.845349 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.845253 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/592cc5de-ba98-448f-9120-4690a117a4ca-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-nzb4f\" (UID: \"592cc5de-ba98-448f-9120-4690a117a4ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f" Jul 22 16:12:55.845349 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.845292 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt72m\" (UniqueName: \"kubernetes.io/projected/592cc5de-ba98-448f-9120-4690a117a4ca-kube-api-access-wt72m\") pod \"kube-storage-version-migrator-operator-7658d9b749-nzb4f\" (UID: \"592cc5de-ba98-448f-9120-4690a117a4ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f" Jul 22 16:12:55.845349 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.845345 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/592cc5de-ba98-448f-9120-4690a117a4ca-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-nzb4f\" (UID: \"592cc5de-ba98-448f-9120-4690a117a4ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f" Jul 22 16:12:55.845897 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.845818 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/592cc5de-ba98-448f-9120-4690a117a4ca-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-nzb4f\" (UID: \"592cc5de-ba98-448f-9120-4690a117a4ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f" Jul 22 16:12:55.847795 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.847773 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/592cc5de-ba98-448f-9120-4690a117a4ca-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-nzb4f\" (UID: \"592cc5de-ba98-448f-9120-4690a117a4ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f" Jul 22 16:12:55.858048 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.858013 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt72m\" (UniqueName: \"kubernetes.io/projected/592cc5de-ba98-448f-9120-4690a117a4ca-kube-api-access-wt72m\") pod \"kube-storage-version-migrator-operator-7658d9b749-nzb4f\" (UID: \"592cc5de-ba98-448f-9120-4690a117a4ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f" Jul 22 16:12:55.877437 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.877411 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6" Jul 22 16:12:55.978776 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:55.978745 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f" Jul 22 16:12:56.007772 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:56.007746 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6"] Jul 22 16:12:56.027780 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:12:56.026782 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02ac6317_0f6c_4fc4_81df_699216b0f136.slice/crio-4636aa6e25c358dd3181655f7f14182b895cd46483a62149a85f76c0cd324990 WatchSource:0}: Error finding container 4636aa6e25c358dd3181655f7f14182b895cd46483a62149a85f76c0cd324990: Status 404 returned error can't find the container with id 4636aa6e25c358dd3181655f7f14182b895cd46483a62149a85f76c0cd324990 Jul 22 16:12:56.119797 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:56.119770 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f"] Jul 22 16:12:56.122738 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:12:56.122713 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod592cc5de_ba98_448f_9120_4690a117a4ca.slice/crio-191ca827ab63a0271e39e5962100af673fccffa4fbbae0937e909edff0fa8571 WatchSource:0}: Error finding container 191ca827ab63a0271e39e5962100af673fccffa4fbbae0937e909edff0fa8571: Status 404 returned error can't find the container with id 191ca827ab63a0271e39e5962100af673fccffa4fbbae0937e909edff0fa8571 Jul 22 16:12:56.834738 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:56.834702 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6" event={"ID":"02ac6317-0f6c-4fc4-81df-699216b0f136","Type":"ContainerStarted","Data":"4636aa6e25c358dd3181655f7f14182b895cd46483a62149a85f76c0cd324990"} Jul 22 16:12:56.836112 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:56.836089 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f" event={"ID":"592cc5de-ba98-448f-9120-4690a117a4ca","Type":"ContainerStarted","Data":"191ca827ab63a0271e39e5962100af673fccffa4fbbae0937e909edff0fa8571"} Jul 22 16:12:57.658623 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:57.658293 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:57.658623 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:57.658408 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:12:57.658623 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:57.658457 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 16:12:57.658623 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:57.658539 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm podName:4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c nodeName:}" failed. No retries permitted until 2026-07-22 16:13:05.658518777 +0000 UTC m=+82.671685797 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mgbrd" (UID: "4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c") : configmap "kube-rbac-proxy" not found Jul 22 16:12:57.658623 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:57.658538 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:12:57.658623 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:12:57.658598 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls podName:4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c nodeName:}" failed. No retries permitted until 2026-07-22 16:13:05.658581993 +0000 UTC m=+82.671749025 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mgbrd" (UID: "4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c") : secret "insights-runtime-extractor-tls" not found Jul 22 16:12:59.847016 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:59.846975 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6" event={"ID":"02ac6317-0f6c-4fc4-81df-699216b0f136","Type":"ContainerStarted","Data":"f484d6a63bfe05b480fd277adcd3c17167eea288746a23f3882b93f4306d43ac"} Jul 22 16:12:59.848575 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:59.848548 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f" event={"ID":"592cc5de-ba98-448f-9120-4690a117a4ca","Type":"ContainerStarted","Data":"2d8da70ae1dc53146aa1ebf87bfd4afdbc5e43bde4fe7f780e3dd5e2a9ab067a"} Jul 22 16:12:59.873150 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:59.873095 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6" podStartSLOduration=2.019473927 podStartE2EDuration="4.873080799s" totalImagesPullingTime="2.853606872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:12:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:12:59.872721875 +0000 UTC m=+76.885888918" watchObservedRunningTime="2026-07-22 16:12:59.873080799 +0000 UTC m=+76.886247840" Jul 22 16:12:59.892174 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:12:59.892125 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f" podStartSLOduration=2.135922408 podStartE2EDuration="4.892110237s" totalImagesPullingTime="2.756187829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:12:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:12:59.891986066 +0000 UTC m=+76.905153106" watchObservedRunningTime="2026-07-22 16:12:59.892110237 +0000 UTC m=+76.905277276" Jul 22 16:13:02.560326 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.560290 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz"] Jul 22 16:13:02.564288 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.564270 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz" Jul 22 16:13:02.567050 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.567029 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 22 16:13:02.567183 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.567164 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 22 16:13:02.567342 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.567322 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6gd9m\"" Jul 22 16:13:02.575812 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.575793 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz"] Jul 22 16:13:02.602300 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.602272 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-5hjsz\" (UID: \"cf370697-0317-4d1d-99c3-f09c1fc2ed4a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz" Jul 22 16:13:02.602416 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.602341 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-5hjsz\" (UID: \"cf370697-0317-4d1d-99c3-f09c1fc2ed4a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz" Jul 22 16:13:02.703575 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.703544 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-5hjsz\" (UID: \"cf370697-0317-4d1d-99c3-f09c1fc2ed4a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz" Jul 22 16:13:02.703745 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.703621 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-5hjsz\" (UID: \"cf370697-0317-4d1d-99c3-f09c1fc2ed4a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz" Jul 22 16:13:02.703745 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:13:02.703741 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 16:13:02.703851 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:13:02.703831 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert podName:cf370697-0317-4d1d-99c3-f09c1fc2ed4a nodeName:}" failed. No retries permitted until 2026-07-22 16:13:03.203809488 +0000 UTC m=+80.216976525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-5hjsz" (UID: "cf370697-0317-4d1d-99c3-f09c1fc2ed4a") : secret "networking-console-plugin-cert" not found Jul 22 16:13:02.704200 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.704180 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-5hjsz\" (UID: \"cf370697-0317-4d1d-99c3-f09c1fc2ed4a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz" Jul 22 16:13:02.761545 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.761513 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-9bx99"] Jul 22 16:13:02.764429 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.764413 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-9bx99" Jul 22 16:13:02.768270 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.768245 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-vfvh5\"" Jul 22 16:13:02.768690 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.768531 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 22 16:13:02.768690 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.768601 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 22 16:13:02.768902 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.768887 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 22 16:13:02.768979 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.768965 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 22 16:13:02.779852 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.779831 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-9bx99"] Jul 22 16:13:02.804142 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.804118 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2a7816c3-68a0-424d-9da4-19679d49b8c6-signing-cabundle\") pod \"service-ca-58598695f7-9bx99\" (UID: \"2a7816c3-68a0-424d-9da4-19679d49b8c6\") " pod="openshift-service-ca/service-ca-58598695f7-9bx99" Jul 22 16:13:02.804231 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.804145 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2a7816c3-68a0-424d-9da4-19679d49b8c6-signing-key\") pod \"service-ca-58598695f7-9bx99\" (UID: \"2a7816c3-68a0-424d-9da4-19679d49b8c6\") " pod="openshift-service-ca/service-ca-58598695f7-9bx99" Jul 22 16:13:02.804231 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.804182 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxk6c\" (UniqueName: \"kubernetes.io/projected/2a7816c3-68a0-424d-9da4-19679d49b8c6-kube-api-access-mxk6c\") pod \"service-ca-58598695f7-9bx99\" (UID: \"2a7816c3-68a0-424d-9da4-19679d49b8c6\") " pod="openshift-service-ca/service-ca-58598695f7-9bx99" Jul 22 16:13:02.905477 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.905440 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2a7816c3-68a0-424d-9da4-19679d49b8c6-signing-cabundle\") pod \"service-ca-58598695f7-9bx99\" (UID: \"2a7816c3-68a0-424d-9da4-19679d49b8c6\") " pod="openshift-service-ca/service-ca-58598695f7-9bx99" Jul 22 16:13:02.905477 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.905476 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2a7816c3-68a0-424d-9da4-19679d49b8c6-signing-key\") pod \"service-ca-58598695f7-9bx99\" (UID: \"2a7816c3-68a0-424d-9da4-19679d49b8c6\") " pod="openshift-service-ca/service-ca-58598695f7-9bx99" Jul 22 16:13:02.905723 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.905703 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxk6c\" (UniqueName: \"kubernetes.io/projected/2a7816c3-68a0-424d-9da4-19679d49b8c6-kube-api-access-mxk6c\") pod \"service-ca-58598695f7-9bx99\" (UID: \"2a7816c3-68a0-424d-9da4-19679d49b8c6\") " pod="openshift-service-ca/service-ca-58598695f7-9bx99" Jul 22 16:13:02.906183 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.906160 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2a7816c3-68a0-424d-9da4-19679d49b8c6-signing-cabundle\") pod \"service-ca-58598695f7-9bx99\" (UID: \"2a7816c3-68a0-424d-9da4-19679d49b8c6\") " pod="openshift-service-ca/service-ca-58598695f7-9bx99" Jul 22 16:13:02.907903 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.907887 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2a7816c3-68a0-424d-9da4-19679d49b8c6-signing-key\") pod \"service-ca-58598695f7-9bx99\" (UID: \"2a7816c3-68a0-424d-9da4-19679d49b8c6\") " pod="openshift-service-ca/service-ca-58598695f7-9bx99" Jul 22 16:13:02.916484 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:02.916458 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxk6c\" (UniqueName: \"kubernetes.io/projected/2a7816c3-68a0-424d-9da4-19679d49b8c6-kube-api-access-mxk6c\") pod \"service-ca-58598695f7-9bx99\" (UID: \"2a7816c3-68a0-424d-9da4-19679d49b8c6\") " pod="openshift-service-ca/service-ca-58598695f7-9bx99" Jul 22 16:13:03.073062 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:03.073028 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-9bx99" Jul 22 16:13:03.209193 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:03.209117 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-5hjsz\" (UID: \"cf370697-0317-4d1d-99c3-f09c1fc2ed4a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz" Jul 22 16:13:03.209331 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:13:03.209254 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 16:13:03.209331 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:13:03.209326 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert podName:cf370697-0317-4d1d-99c3-f09c1fc2ed4a nodeName:}" failed. No retries permitted until 2026-07-22 16:13:04.209306077 +0000 UTC m=+81.222473111 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-5hjsz" (UID: "cf370697-0317-4d1d-99c3-f09c1fc2ed4a") : secret "networking-console-plugin-cert" not found Jul 22 16:13:03.213854 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:03.213830 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-9bx99"] Jul 22 16:13:03.216715 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:13:03.216690 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a7816c3_68a0_424d_9da4_19679d49b8c6.slice/crio-e6471c9ab9b39b52d9a4233caad3924a7828e7a8c07d53f49cbd240c84b9ed24 WatchSource:0}: Error finding container e6471c9ab9b39b52d9a4233caad3924a7828e7a8c07d53f49cbd240c84b9ed24: Status 404 returned error can't find the container with id e6471c9ab9b39b52d9a4233caad3924a7828e7a8c07d53f49cbd240c84b9ed24 Jul 22 16:13:03.860357 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:03.860316 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-9bx99" event={"ID":"2a7816c3-68a0-424d-9da4-19679d49b8c6","Type":"ContainerStarted","Data":"a568deda58f968c33bc8b101a8d22199156fe430d42705c49b5a8434aa4bae78"} Jul 22 16:13:03.860827 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:03.860419 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-9bx99" event={"ID":"2a7816c3-68a0-424d-9da4-19679d49b8c6","Type":"ContainerStarted","Data":"e6471c9ab9b39b52d9a4233caad3924a7828e7a8c07d53f49cbd240c84b9ed24"} Jul 22 16:13:03.883799 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:03.883749 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-9bx99" podStartSLOduration=1.883733398 podStartE2EDuration="1.883733398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:03.883135638 +0000 UTC m=+80.896302678" watchObservedRunningTime="2026-07-22 16:13:03.883733398 +0000 UTC m=+80.896900446" Jul 22 16:13:04.215976 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:04.215923 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-5hjsz\" (UID: \"cf370697-0317-4d1d-99c3-f09c1fc2ed4a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz" Jul 22 16:13:04.216220 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:13:04.216107 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 16:13:04.216220 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:13:04.216200 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert podName:cf370697-0317-4d1d-99c3-f09c1fc2ed4a nodeName:}" failed. No retries permitted until 2026-07-22 16:13:06.216176751 +0000 UTC m=+83.229343768 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-5hjsz" (UID: "cf370697-0317-4d1d-99c3-f09c1fc2ed4a") : secret "networking-console-plugin-cert" not found Jul 22 16:13:05.729060 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:05.728936 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:13:05.729060 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:05.728981 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:13:05.729599 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:13:05.729181 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 16:13:05.729599 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:13:05.729281 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls podName:4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c nodeName:}" failed. No retries permitted until 2026-07-22 16:13:21.729257015 +0000 UTC m=+98.742424031 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mgbrd" (UID: "4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c") : secret "insights-runtime-extractor-tls" not found Jul 22 16:13:05.729599 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:05.729591 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:13:06.234246 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:06.234206 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-5hjsz\" (UID: \"cf370697-0317-4d1d-99c3-f09c1fc2ed4a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz" Jul 22 16:13:06.234437 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:13:06.234355 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 16:13:06.234486 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:13:06.234467 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert podName:cf370697-0317-4d1d-99c3-f09c1fc2ed4a nodeName:}" failed. No retries permitted until 2026-07-22 16:13:10.234444827 +0000 UTC m=+87.247611855 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-5hjsz" (UID: "cf370697-0317-4d1d-99c3-f09c1fc2ed4a") : secret "networking-console-plugin-cert" not found Jul 22 16:13:10.267510 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:10.267474 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-5hjsz\" (UID: \"cf370697-0317-4d1d-99c3-f09c1fc2ed4a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz" Jul 22 16:13:10.267935 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:13:10.267642 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 16:13:10.267935 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:13:10.267709 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert podName:cf370697-0317-4d1d-99c3-f09c1fc2ed4a nodeName:}" failed. No retries permitted until 2026-07-22 16:13:18.267691976 +0000 UTC m=+95.280858995 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-5hjsz" (UID: "cf370697-0317-4d1d-99c3-f09c1fc2ed4a") : secret "networking-console-plugin-cert" not found Jul 22 16:13:18.332445 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:18.332372 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-5hjsz\" (UID: \"cf370697-0317-4d1d-99c3-f09c1fc2ed4a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz" Jul 22 16:13:18.334858 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:18.334836 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cf370697-0317-4d1d-99c3-f09c1fc2ed4a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-5hjsz\" (UID: \"cf370697-0317-4d1d-99c3-f09c1fc2ed4a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz" Jul 22 16:13:18.472490 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:18.472461 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz" Jul 22 16:13:18.600664 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:18.600586 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz"] Jul 22 16:13:18.603331 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:13:18.603293 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf370697_0317_4d1d_99c3_f09c1fc2ed4a.slice/crio-994b1467f0f96c050477a48248d5df1919e0a16ae6f66a4ce89ad4723ee36d41 WatchSource:0}: Error finding container 994b1467f0f96c050477a48248d5df1919e0a16ae6f66a4ce89ad4723ee36d41: Status 404 returned error can't find the container with id 994b1467f0f96c050477a48248d5df1919e0a16ae6f66a4ce89ad4723ee36d41 Jul 22 16:13:18.911711 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:18.911676 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz" event={"ID":"cf370697-0317-4d1d-99c3-f09c1fc2ed4a","Type":"ContainerStarted","Data":"994b1467f0f96c050477a48248d5df1919e0a16ae6f66a4ce89ad4723ee36d41"} Jul 22 16:13:20.921027 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:20.920994 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz" event={"ID":"cf370697-0317-4d1d-99c3-f09c1fc2ed4a","Type":"ContainerStarted","Data":"98c655df206d59509e109a9ede39e89a1d09197d6069a5a6fcd98ecf9c5ec41b"} Jul 22 16:13:20.942643 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:20.942594 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5hjsz" podStartSLOduration=17.695263756 podStartE2EDuration="18.942579495s" totalImagesPullingTime="1.247315739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:20.941512455 +0000 UTC m=+97.954679522" watchObservedRunningTime="2026-07-22 16:13:20.942579495 +0000 UTC m=+97.955746566" Jul 22 16:13:21.760010 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:21.759976 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:13:21.762173 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:21.762145 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mgbrd\" (UID: \"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c\") " pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:13:21.790037 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:21.790013 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mgbrd" Jul 22 16:13:21.924750 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:21.924705 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mgbrd"] Jul 22 16:13:21.927870 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:13:21.927837 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d7af12d_36fa_4ff9_b6d9_0dbae4e0bc3c.slice/crio-2839f18a245073f87ba3063b32fd75871bbf0cf343dfc6c3b3d64f37d82c6eb0 WatchSource:0}: Error finding container 2839f18a245073f87ba3063b32fd75871bbf0cf343dfc6c3b3d64f37d82c6eb0: Status 404 returned error can't find the container with id 2839f18a245073f87ba3063b32fd75871bbf0cf343dfc6c3b3d64f37d82c6eb0 Jul 22 16:13:22.768086 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:22.768049 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:13:22.768259 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:22.768143 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:13:22.771103 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:22.771067 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls\") pod \"image-registry-5f8ffcd8f4-666kp\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:13:22.771730 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:22.771706 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c228a9d-7811-459b-8b7a-6dd6e6f5a76a-metrics-tls\") pod \"dns-default-m9wx5\" (UID: \"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a\") " pod="openshift-dns/dns-default-m9wx5" Jul 22 16:13:22.774962 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:22.774941 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bj2bw\"" Jul 22 16:13:22.782825 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:22.782799 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m9wx5" Jul 22 16:13:22.928868 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:22.928834 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mgbrd" event={"ID":"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c","Type":"ContainerStarted","Data":"2839f18a245073f87ba3063b32fd75871bbf0cf343dfc6c3b3d64f37d82c6eb0"} Jul 22 16:13:22.943193 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:22.943120 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m9wx5"] Jul 22 16:13:22.947783 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:13:22.947748 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c228a9d_7811_459b_8b7a_6dd6e6f5a76a.slice/crio-0027e7b55f7c33dfe0fa393d9979dffe119b4bc4763fcfdc21bae138a9ae085a WatchSource:0}: Error finding container 0027e7b55f7c33dfe0fa393d9979dffe119b4bc4763fcfdc21bae138a9ae085a: Status 404 returned error can't find the container with id 0027e7b55f7c33dfe0fa393d9979dffe119b4bc4763fcfdc21bae138a9ae085a Jul 22 16:13:23.050790 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:23.050758 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-lcgb9\"" Jul 22 16:13:23.058478 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:23.058214 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:13:23.224465 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:23.224430 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f8ffcd8f4-666kp"] Jul 22 16:13:23.228732 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:13:23.228701 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26e2f0f2_77f1_45d0_88b5_bb10285392d0.slice/crio-0c34af9022e9a675a0b8c997d21379e950eec659d3e541e26fe6276a937415f5 WatchSource:0}: Error finding container 0c34af9022e9a675a0b8c997d21379e950eec659d3e541e26fe6276a937415f5: Status 404 returned error can't find the container with id 0c34af9022e9a675a0b8c997d21379e950eec659d3e541e26fe6276a937415f5 Jul 22 16:13:23.933950 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:23.933908 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" event={"ID":"26e2f0f2-77f1-45d0-88b5-bb10285392d0","Type":"ContainerStarted","Data":"d57a5c405d09431b38def6f4d5e18ade0fc69280df2faf0b3a4e443a6e2af7af"} Jul 22 16:13:23.934424 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:23.933961 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" event={"ID":"26e2f0f2-77f1-45d0-88b5-bb10285392d0","Type":"ContainerStarted","Data":"0c34af9022e9a675a0b8c997d21379e950eec659d3e541e26fe6276a937415f5"} Jul 22 16:13:23.934424 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:23.934062 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:13:23.935111 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:23.935085 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m9wx5" event={"ID":"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a","Type":"ContainerStarted","Data":"0027e7b55f7c33dfe0fa393d9979dffe119b4bc4763fcfdc21bae138a9ae085a"} Jul 22 16:13:25.832566 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:25.832532 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-blvd8" Jul 22 16:13:25.857659 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:25.857607 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" podStartSLOduration=101.85758813 podStartE2EDuration="1m41.85758813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:11:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:23.964359741 +0000 UTC m=+100.977526781" watchObservedRunningTime="2026-07-22 16:13:25.85758813 +0000 UTC m=+102.870755172" Jul 22 16:13:26.533025 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:26.532887 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d7fd97d66-tvspc"] Jul 22 16:13:26.722054 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:26.722028 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d7fd97d66-tvspc"] Jul 22 16:13:26.722054 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:26.722058 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dpt7j"] Jul 22 16:13:26.722255 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:26.722173 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d7fd97d66-tvspc" Jul 22 16:13:26.725560 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:26.725538 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-dfwsh\"" Jul 22 16:13:26.725729 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:26.725709 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 22 16:13:26.726718 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:26.726699 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 22 16:13:26.726811 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:26.726722 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 22 16:13:26.727084 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:26.727070 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 22 16:13:26.763173 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:26.763153 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dpt7j"] Jul 22 16:13:26.763269 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:26.763243 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dpt7j" Jul 22 16:13:26.768071 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:26.768053 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-tfw2h\"" Jul 22 16:13:26.768357 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:26.768340 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 22 16:13:26.903296 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:26.903263 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/738e099a-c837-4176-9a84-04e5414a2959-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-dpt7j\" (UID: \"738e099a-c837-4176-9a84-04e5414a2959\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dpt7j" Jul 22 16:13:26.903674 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:26.903299 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sczcz\" (UniqueName: \"kubernetes.io/projected/df0b4ed7-386b-43bd-a77a-26d3bc742e51-kube-api-access-sczcz\") pod \"managed-serviceaccount-addon-agent-6d7fd97d66-tvspc\" (UID: \"df0b4ed7-386b-43bd-a77a-26d3bc742e51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d7fd97d66-tvspc" Jul 22 16:13:26.903674 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:26.903371 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/df0b4ed7-386b-43bd-a77a-26d3bc742e51-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d7fd97d66-tvspc\" (UID: \"df0b4ed7-386b-43bd-a77a-26d3bc742e51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d7fd97d66-tvspc" Jul 22 16:13:27.003979 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:27.003957 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/738e099a-c837-4176-9a84-04e5414a2959-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-dpt7j\" (UID: \"738e099a-c837-4176-9a84-04e5414a2959\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dpt7j" Jul 22 16:13:27.004076 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:27.003988 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sczcz\" (UniqueName: \"kubernetes.io/projected/df0b4ed7-386b-43bd-a77a-26d3bc742e51-kube-api-access-sczcz\") pod \"managed-serviceaccount-addon-agent-6d7fd97d66-tvspc\" (UID: \"df0b4ed7-386b-43bd-a77a-26d3bc742e51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d7fd97d66-tvspc" Jul 22 16:13:27.004076 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:27.004041 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/df0b4ed7-386b-43bd-a77a-26d3bc742e51-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d7fd97d66-tvspc\" (UID: \"df0b4ed7-386b-43bd-a77a-26d3bc742e51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d7fd97d66-tvspc" Jul 22 16:13:27.006563 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:27.006542 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/df0b4ed7-386b-43bd-a77a-26d3bc742e51-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6d7fd97d66-tvspc\" (UID: \"df0b4ed7-386b-43bd-a77a-26d3bc742e51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d7fd97d66-tvspc" Jul 22 16:13:27.006677 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:27.006570 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/738e099a-c837-4176-9a84-04e5414a2959-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-dpt7j\" (UID: \"738e099a-c837-4176-9a84-04e5414a2959\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dpt7j" Jul 22 16:13:27.025808 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:27.025786 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sczcz\" (UniqueName: \"kubernetes.io/projected/df0b4ed7-386b-43bd-a77a-26d3bc742e51-kube-api-access-sczcz\") pod \"managed-serviceaccount-addon-agent-6d7fd97d66-tvspc\" (UID: \"df0b4ed7-386b-43bd-a77a-26d3bc742e51\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d7fd97d66-tvspc" Jul 22 16:13:27.049621 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:27.049583 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d7fd97d66-tvspc" Jul 22 16:13:27.071183 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:27.071160 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dpt7j" Jul 22 16:13:27.972426 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:27.972356 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dpt7j"] Jul 22 16:13:27.994253 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:27.994216 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d7fd97d66-tvspc"] Jul 22 16:13:28.092059 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:13:28.091973 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod738e099a_c837_4176_9a84_04e5414a2959.slice/crio-db43e1b2852401563a293e7ba76db03642e29f30e751b746e6f1299db0e426e5 WatchSource:0}: Error finding container db43e1b2852401563a293e7ba76db03642e29f30e751b746e6f1299db0e426e5: Status 404 returned error can't find the container with id db43e1b2852401563a293e7ba76db03642e29f30e751b746e6f1299db0e426e5 Jul 22 16:13:28.092733 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:13:28.092706 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf0b4ed7_386b_43bd_a77a_26d3bc742e51.slice/crio-10b4dc616ecba6df151d0f6488da723ad3f442cb5f10ad111f32f2d37bd698da WatchSource:0}: Error finding container 10b4dc616ecba6df151d0f6488da723ad3f442cb5f10ad111f32f2d37bd698da: Status 404 returned error can't find the container with id 10b4dc616ecba6df151d0f6488da723ad3f442cb5f10ad111f32f2d37bd698da Jul 22 16:13:28.956118 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:28.956069 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mgbrd" event={"ID":"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c","Type":"ContainerStarted","Data":"994bd841e12fc3fa7c3d656c1a0db27d652ff255f432c05fd6af15e638b5c5c0"} Jul 22 16:13:28.958056 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:28.958004 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m9wx5" event={"ID":"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a","Type":"ContainerStarted","Data":"70dfa319a7b89aeef9856e69dba4d323b0ab5fdf8237877be557f3d2bc6396e1"} Jul 22 16:13:28.958056 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:28.958057 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m9wx5" event={"ID":"4c228a9d-7811-459b-8b7a-6dd6e6f5a76a","Type":"ContainerStarted","Data":"9ef44ba3469c687c891628689aaf48f0cd5aac9411025fa2841d46b55dd0983f"} Jul 22 16:13:28.958240 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:28.958103 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-m9wx5" Jul 22 16:13:28.959600 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:28.959575 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d7fd97d66-tvspc" event={"ID":"df0b4ed7-386b-43bd-a77a-26d3bc742e51","Type":"ContainerStarted","Data":"10b4dc616ecba6df151d0f6488da723ad3f442cb5f10ad111f32f2d37bd698da"} Jul 22 16:13:28.960770 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:28.960750 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dpt7j" event={"ID":"738e099a-c837-4176-9a84-04e5414a2959","Type":"ContainerStarted","Data":"db43e1b2852401563a293e7ba76db03642e29f30e751b746e6f1299db0e426e5"} Jul 22 16:13:28.996502 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:28.996459 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-m9wx5" podStartSLOduration=66.158563171 podStartE2EDuration="1m10.996446613s" totalImagesPullingTime="4.837883442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:28.995582113 +0000 UTC m=+106.008749152" watchObservedRunningTime="2026-07-22 16:13:28.996446613 +0000 UTC m=+106.009613646" Jul 22 16:13:30.970672 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:30.970630 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dpt7j" event={"ID":"738e099a-c837-4176-9a84-04e5414a2959","Type":"ContainerStarted","Data":"576495d876bc2d0f39b835e0b8fc9d745fdb1ddbe5f7b17d92ea52c71c8a00f4"} Jul 22 16:13:30.971028 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:30.970810 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dpt7j" Jul 22 16:13:30.975852 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:30.975820 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dpt7j" Jul 22 16:13:30.999526 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:30.999478 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-dpt7j" podStartSLOduration=3.141346343 podStartE2EDuration="4.999464288s" totalImagesPullingTime="1.858117945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:30.998903786 +0000 UTC m=+108.012070826" watchObservedRunningTime="2026-07-22 16:13:30.999464288 +0000 UTC m=+108.012631326" Jul 22 16:13:31.793155 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:31.793121 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj"] Jul 22 16:13:31.803429 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:31.803380 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" Jul 22 16:13:31.806416 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:31.806374 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 22 16:13:31.807894 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:31.807686 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 16:13:31.807894 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:31.807711 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-wfv4b\"" Jul 22 16:13:31.807894 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:31.807724 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 22 16:13:31.807894 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:31.807728 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 16:13:31.807894 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:31.807816 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 16:13:31.809665 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:31.809205 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj"] Jul 22 16:13:31.942670 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:31.942547 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae4ea54d-b612-480c-8415-c13e53c2588b-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-jvbvj\" (UID: \"ae4ea54d-b612-480c-8415-c13e53c2588b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" Jul 22 16:13:31.942670 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:31.942656 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8s26\" (UniqueName: \"kubernetes.io/projected/ae4ea54d-b612-480c-8415-c13e53c2588b-kube-api-access-w8s26\") pod \"prometheus-operator-5945b5cfd-jvbvj\" (UID: \"ae4ea54d-b612-480c-8415-c13e53c2588b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" Jul 22 16:13:31.942886 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:31.942716 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae4ea54d-b612-480c-8415-c13e53c2588b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-jvbvj\" (UID: \"ae4ea54d-b612-480c-8415-c13e53c2588b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" Jul 22 16:13:31.942886 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:31.942744 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae4ea54d-b612-480c-8415-c13e53c2588b-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-jvbvj\" (UID: \"ae4ea54d-b612-480c-8415-c13e53c2588b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" Jul 22 16:13:32.043850 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:32.043770 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae4ea54d-b612-480c-8415-c13e53c2588b-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-jvbvj\" (UID: \"ae4ea54d-b612-480c-8415-c13e53c2588b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" Jul 22 16:13:32.043850 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:32.043834 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8s26\" (UniqueName: \"kubernetes.io/projected/ae4ea54d-b612-480c-8415-c13e53c2588b-kube-api-access-w8s26\") pod \"prometheus-operator-5945b5cfd-jvbvj\" (UID: \"ae4ea54d-b612-480c-8415-c13e53c2588b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" Jul 22 16:13:32.044310 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:32.043876 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae4ea54d-b612-480c-8415-c13e53c2588b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-jvbvj\" (UID: \"ae4ea54d-b612-480c-8415-c13e53c2588b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" Jul 22 16:13:32.044310 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:32.043908 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae4ea54d-b612-480c-8415-c13e53c2588b-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-jvbvj\" (UID: \"ae4ea54d-b612-480c-8415-c13e53c2588b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" Jul 22 16:13:32.044310 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:13:32.043924 2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 22 16:13:32.044310 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:13:32.043996 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae4ea54d-b612-480c-8415-c13e53c2588b-prometheus-operator-tls podName:ae4ea54d-b612-480c-8415-c13e53c2588b nodeName:}" failed. No retries permitted until 2026-07-22 16:13:32.543976616 +0000 UTC m=+109.557143635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/ae4ea54d-b612-480c-8415-c13e53c2588b-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-jvbvj" (UID: "ae4ea54d-b612-480c-8415-c13e53c2588b") : secret "prometheus-operator-tls" not found Jul 22 16:13:32.044646 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:32.044628 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae4ea54d-b612-480c-8415-c13e53c2588b-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-jvbvj\" (UID: \"ae4ea54d-b612-480c-8415-c13e53c2588b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" Jul 22 16:13:32.046788 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:32.046766 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae4ea54d-b612-480c-8415-c13e53c2588b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-jvbvj\" (UID: \"ae4ea54d-b612-480c-8415-c13e53c2588b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" Jul 22 16:13:32.057602 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:32.057576 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8s26\" (UniqueName: \"kubernetes.io/projected/ae4ea54d-b612-480c-8415-c13e53c2588b-kube-api-access-w8s26\") pod \"prometheus-operator-5945b5cfd-jvbvj\" (UID: \"ae4ea54d-b612-480c-8415-c13e53c2588b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" Jul 22 16:13:32.547818 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:32.547788 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae4ea54d-b612-480c-8415-c13e53c2588b-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-jvbvj\" (UID: \"ae4ea54d-b612-480c-8415-c13e53c2588b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" Jul 22 16:13:32.550111 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:32.550091 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ae4ea54d-b612-480c-8415-c13e53c2588b-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-jvbvj\" (UID: \"ae4ea54d-b612-480c-8415-c13e53c2588b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" Jul 22 16:13:32.715684 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:32.715650 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" Jul 22 16:13:33.379466 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:33.379326 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj"] Jul 22 16:13:33.381922 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:13:33.381899 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae4ea54d_b612_480c_8415_c13e53c2588b.slice/crio-25c53468287dbb10a15d429a145e46ee87aa9d7d7d0a0b7eb0dabd0fe576afdb WatchSource:0}: Error finding container 25c53468287dbb10a15d429a145e46ee87aa9d7d7d0a0b7eb0dabd0fe576afdb: Status 404 returned error can't find the container with id 25c53468287dbb10a15d429a145e46ee87aa9d7d7d0a0b7eb0dabd0fe576afdb Jul 22 16:13:33.984668 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:33.984626 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d7fd97d66-tvspc" event={"ID":"df0b4ed7-386b-43bd-a77a-26d3bc742e51","Type":"ContainerStarted","Data":"b72feb591510a7d47fac9e26ba4c562a638378589eebc753f10f0ebb18147311"} Jul 22 16:13:33.986877 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:33.986847 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mgbrd" event={"ID":"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c","Type":"ContainerStarted","Data":"0d94de15cf1c8df76feb7cc17f61d1b6c471a079bc72f7f2bde4bd35404c0509"} Jul 22 16:13:33.988243 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:33.988198 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" event={"ID":"ae4ea54d-b612-480c-8415-c13e53c2588b","Type":"ContainerStarted","Data":"25c53468287dbb10a15d429a145e46ee87aa9d7d7d0a0b7eb0dabd0fe576afdb"} Jul 22 16:13:34.004432 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:34.004360 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6d7fd97d66-tvspc" podStartSLOduration=2.879583865 podStartE2EDuration="8.004342421s" totalImagesPullingTime="5.124758556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:34.003356332 +0000 UTC m=+111.016523368" watchObservedRunningTime="2026-07-22 16:13:34.004342421 +0000 UTC m=+111.017509460" Jul 22 16:13:34.995270 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:34.995231 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" event={"ID":"ae4ea54d-b612-480c-8415-c13e53c2588b","Type":"ContainerStarted","Data":"3e8baa20fa8a979194c17d906c4193310c380924031b0d04d82422322945fe07"} Jul 22 16:13:34.995659 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:34.995302 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" event={"ID":"ae4ea54d-b612-480c-8415-c13e53c2588b","Type":"ContainerStarted","Data":"aa95d21df428ba956e874b5fdaa659a32e843d96e01451ea98bc74edc75c52e0"} Jul 22 16:13:35.029629 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:35.029532 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jvbvj" podStartSLOduration=2.657681515 podStartE2EDuration="4.02951201s" totalImagesPullingTime="1.371830495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:35.028106 +0000 UTC m=+112.041273041" watchObservedRunningTime="2026-07-22 16:13:35.02951201 +0000 UTC m=+112.042679050" Jul 22 16:13:37.004942 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.004916 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mgbrd" event={"ID":"4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c","Type":"ContainerStarted","Data":"93e6087d80e669e96b0077776955efa788c0e67a5e82c2e529d015a31b26790b"} Jul 22 16:13:37.564975 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.564916 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mgbrd" podStartSLOduration=33.755100783 podStartE2EDuration="48.564896751s" totalImagesPullingTime="14.809795968s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:12:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:37.047132254 +0000 UTC m=+114.060299296" watchObservedRunningTime="2026-07-22 16:13:37.564896751 +0000 UTC m=+114.578063791" Jul 22 16:13:37.566110 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.566090 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28"] Jul 22 16:13:37.569566 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.569548 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" Jul 22 16:13:37.572970 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.572949 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 22 16:13:37.573623 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.573486 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-2qfl5\"" Jul 22 16:13:37.573623 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.573618 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 22 16:13:37.587789 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.587768 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e212f29c-872c-45ec-a580-73ac0e757d9b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-t7p28\" (UID: \"e212f29c-872c-45ec-a580-73ac0e757d9b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" Jul 22 16:13:37.587890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.587808 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzk8x\" (UniqueName: \"kubernetes.io/projected/e212f29c-872c-45ec-a580-73ac0e757d9b-kube-api-access-kzk8x\") pod \"openshift-state-metrics-79f6958cdf-t7p28\" (UID: \"e212f29c-872c-45ec-a580-73ac0e757d9b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" Jul 22 16:13:37.587890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.587829 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e212f29c-872c-45ec-a580-73ac0e757d9b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-t7p28\" (UID: \"e212f29c-872c-45ec-a580-73ac0e757d9b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" Jul 22 16:13:37.587890 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.587844 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e212f29c-872c-45ec-a580-73ac0e757d9b-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-t7p28\" (UID: \"e212f29c-872c-45ec-a580-73ac0e757d9b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" Jul 22 16:13:37.684663 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.684642 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5shsb"] Jul 22 16:13:37.687703 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.687684 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28"] Jul 22 16:13:37.687815 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.687801 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.688245 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.688222 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq5bp\" (UniqueName: \"kubernetes.io/projected/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-kube-api-access-xq5bp\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.688334 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.688254 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-sys\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.688334 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.688281 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.688334 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.688315 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e212f29c-872c-45ec-a580-73ac0e757d9b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-t7p28\" (UID: \"e212f29c-872c-45ec-a580-73ac0e757d9b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" Jul 22 16:13:37.688508 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.688342 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-root\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.688508 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.688362 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzk8x\" (UniqueName: \"kubernetes.io/projected/e212f29c-872c-45ec-a580-73ac0e757d9b-kube-api-access-kzk8x\") pod \"openshift-state-metrics-79f6958cdf-t7p28\" (UID: \"e212f29c-872c-45ec-a580-73ac0e757d9b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" Jul 22 16:13:37.688508 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.688382 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-metrics-client-ca\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.688508 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.688441 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e212f29c-872c-45ec-a580-73ac0e757d9b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-t7p28\" (UID: \"e212f29c-872c-45ec-a580-73ac0e757d9b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" Jul 22 16:13:37.688508 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.688461 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e212f29c-872c-45ec-a580-73ac0e757d9b-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-t7p28\" (UID: \"e212f29c-872c-45ec-a580-73ac0e757d9b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" Jul 22 16:13:37.688508 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.688486 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-node-exporter-textfile\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.688787 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.688520 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-node-exporter-tls\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.688787 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.688542 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-node-exporter-wtmp\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.688787 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.688563 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-node-exporter-accelerators-collector-config\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.689425 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.689405 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e212f29c-872c-45ec-a580-73ac0e757d9b-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-t7p28\" (UID: \"e212f29c-872c-45ec-a580-73ac0e757d9b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" Jul 22 16:13:37.690881 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.690858 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e212f29c-872c-45ec-a580-73ac0e757d9b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-t7p28\" (UID: \"e212f29c-872c-45ec-a580-73ac0e757d9b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" Jul 22 16:13:37.691160 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.691146 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e212f29c-872c-45ec-a580-73ac0e757d9b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-t7p28\" (UID: \"e212f29c-872c-45ec-a580-73ac0e757d9b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" Jul 22 16:13:37.695655 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.695620 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 16:13:37.695759 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.695741 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ll44p\"" Jul 22 16:13:37.696223 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.696199 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 16:13:37.729226 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.729191 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzk8x\" (UniqueName: \"kubernetes.io/projected/e212f29c-872c-45ec-a580-73ac0e757d9b-kube-api-access-kzk8x\") pod \"openshift-state-metrics-79f6958cdf-t7p28\" (UID: \"e212f29c-872c-45ec-a580-73ac0e757d9b\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" Jul 22 16:13:37.789545 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.789516 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-root\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.789663 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.789560 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-metrics-client-ca\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.789663 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.789598 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-node-exporter-textfile\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.789663 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.789622 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-root\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.789663 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.789628 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-node-exporter-tls\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.789882 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.789679 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-node-exporter-wtmp\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.789882 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.789708 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-node-exporter-accelerators-collector-config\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.789882 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.789755 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq5bp\" (UniqueName: \"kubernetes.io/projected/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-kube-api-access-xq5bp\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.789882 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.789781 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-sys\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.789882 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.789822 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.789882 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.789836 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-node-exporter-wtmp\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.790192 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.789875 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-sys\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.790192 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.790065 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-node-exporter-textfile\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.790303 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.790210 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-metrics-client-ca\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.790355 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.790309 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-node-exporter-accelerators-collector-config\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.792164 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.792142 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.856317 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.856252 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq5bp\" (UniqueName: \"kubernetes.io/projected/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-kube-api-access-xq5bp\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:37.880345 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.880316 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" Jul 22 16:13:37.880549 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.880529 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 16:13:37.892972 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:37.892943 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/588ab578-e02f-44ab-98c5-5ca5b4b9e35c-node-exporter-tls\") pod \"node-exporter-5shsb\" (UID: \"588ab578-e02f-44ab-98c5-5ca5b4b9e35c\") " pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:38.004883 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:38.004838 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5shsb" Jul 22 16:13:38.016322 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:13:38.016295 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod588ab578_e02f_44ab_98c5_5ca5b4b9e35c.slice/crio-9814c987bef34545cdf225b1907200752439448b0eae58cd18fa7c76d89929d5 WatchSource:0}: Error finding container 9814c987bef34545cdf225b1907200752439448b0eae58cd18fa7c76d89929d5: Status 404 returned error can't find the container with id 9814c987bef34545cdf225b1907200752439448b0eae58cd18fa7c76d89929d5 Jul 22 16:13:38.057127 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:38.057101 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28"] Jul 22 16:13:38.059917 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:13:38.059894 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode212f29c_872c_45ec_a580_73ac0e757d9b.slice/crio-03c94d1c691f5d126f837082b9a9d4052073c7dc34d6bfdeddefe3036d36079c WatchSource:0}: Error finding container 03c94d1c691f5d126f837082b9a9d4052073c7dc34d6bfdeddefe3036d36079c: Status 404 returned error can't find the container with id 03c94d1c691f5d126f837082b9a9d4052073c7dc34d6bfdeddefe3036d36079c Jul 22 16:13:38.968064 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:38.968039 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-m9wx5" Jul 22 16:13:39.015333 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:39.015295 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" event={"ID":"e212f29c-872c-45ec-a580-73ac0e757d9b","Type":"ContainerStarted","Data":"d15ba8fcf8187d0dfadb6f427b17f9d4413157bf8f44719e66261e2f02e1a7c1"} Jul 22 16:13:39.015487 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:39.015377 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" event={"ID":"e212f29c-872c-45ec-a580-73ac0e757d9b","Type":"ContainerStarted","Data":"1282dbd7d9dd95db3159b133be7d6c56652c2dfbeb892995c6036749b04c2717"} Jul 22 16:13:39.015487 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:39.015445 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" event={"ID":"e212f29c-872c-45ec-a580-73ac0e757d9b","Type":"ContainerStarted","Data":"03c94d1c691f5d126f837082b9a9d4052073c7dc34d6bfdeddefe3036d36079c"} Jul 22 16:13:39.017582 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:39.017526 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5shsb" event={"ID":"588ab578-e02f-44ab-98c5-5ca5b4b9e35c","Type":"ContainerStarted","Data":"9814c987bef34545cdf225b1907200752439448b0eae58cd18fa7c76d89929d5"} Jul 22 16:13:40.026674 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:40.026631 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" event={"ID":"e212f29c-872c-45ec-a580-73ac0e757d9b","Type":"ContainerStarted","Data":"71ea4c32c828e91de6831d70933b68f0d901dcaad40d97ee69ff24fa1474d18f"} Jul 22 16:13:40.028231 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:40.028208 2565 generic.go:356] "Generic (PLEG): container finished" podID="588ab578-e02f-44ab-98c5-5ca5b4b9e35c" containerID="6f729e3f03f6c7c7074854e4b8f85b6b457b17987e9a8ac7eedc1c2fc8eaef7f" exitCode=0 Jul 22 16:13:40.028328 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:40.028250 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5shsb" event={"ID":"588ab578-e02f-44ab-98c5-5ca5b4b9e35c","Type":"ContainerDied","Data":"6f729e3f03f6c7c7074854e4b8f85b6b457b17987e9a8ac7eedc1c2fc8eaef7f"} Jul 22 16:13:40.063884 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:40.063828 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-t7p28" podStartSLOduration=1.773218228 podStartE2EDuration="3.063809545s" totalImagesPullingTime="1.290591317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:40.061291482 +0000 UTC m=+117.074458520" watchObservedRunningTime="2026-07-22 16:13:40.063809545 +0000 UTC m=+117.076976585" Jul 22 16:13:41.034458 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:41.034421 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5shsb" event={"ID":"588ab578-e02f-44ab-98c5-5ca5b4b9e35c","Type":"ContainerStarted","Data":"0dd6a8841a082e096048c28e10c23aff2666ff9521a18f1e9ae7376346464575"} Jul 22 16:13:41.034825 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:41.034474 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5shsb" event={"ID":"588ab578-e02f-44ab-98c5-5ca5b4b9e35c","Type":"ContainerStarted","Data":"4fc21dfb3a4f13ade4710714253338fa5c6a3fb841d32be5cc2a89723695e4c0"} Jul 22 16:13:41.079467 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:41.079356 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5shsb" podStartSLOduration=3.056958199 podStartE2EDuration="4.079323862s" totalImagesPullingTime="947.521257ms" totalInitContainerRuntime="74.844406ms" isStatefulPod=true podCreationTimestamp="2026-07-22 16:13:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:41.077684541 +0000 UTC m=+118.090851610" watchObservedRunningTime="2026-07-22 16:13:41.079323862 +0000 UTC m=+118.092490892" Jul 22 16:13:42.457703 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.457666 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-994875d55-g2w9s"] Jul 22 16:13:42.461304 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.461283 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.470262 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.470238 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 22 16:13:42.470382 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.470239 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 22 16:13:42.470382 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.470337 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 22 16:13:42.470532 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.470246 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 22 16:13:42.470532 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.470250 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-kz6lx\"" Jul 22 16:13:42.472858 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.472843 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7l9m2gpl43fsp\"" Jul 22 16:13:42.491869 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.491838 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-994875d55-g2w9s"] Jul 22 16:13:42.531148 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.531123 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-client-ca-bundle\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.531283 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.531154 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-metrics-server-audit-profiles\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.531283 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.531187 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-secret-metrics-server-tls\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.531423 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.531306 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wvwp\" (UniqueName: \"kubernetes.io/projected/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-kube-api-access-6wvwp\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.531423 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.531375 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-secret-metrics-server-client-certs\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.531527 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.531430 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.531527 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.531454 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-audit-log\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.632457 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.632419 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-client-ca-bundle\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.632626 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.632463 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-metrics-server-audit-profiles\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.632697 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.632627 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-secret-metrics-server-tls\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.632753 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.632725 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wvwp\" (UniqueName: \"kubernetes.io/projected/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-kube-api-access-6wvwp\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.632807 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.632795 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-secret-metrics-server-client-certs\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.632935 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.632911 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.632935 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.632943 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-audit-log\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.633281 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.633261 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-audit-log\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.633782 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.633756 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-metrics-server-audit-profiles\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.633946 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.633919 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.635328 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.635308 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-client-ca-bundle\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.635452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.635434 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-secret-metrics-server-tls\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.635792 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.635774 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-secret-metrics-server-client-certs\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.660523 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.660500 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wvwp\" (UniqueName: \"kubernetes.io/projected/dd837a3a-0d04-4cb7-9812-b0f1e306a3b3-kube-api-access-6wvwp\") pod \"metrics-server-994875d55-g2w9s\" (UID: \"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3\") " pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.770351 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.770277 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:13:42.903912 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:42.903880 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-994875d55-g2w9s"] Jul 22 16:13:42.906705 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:13:42.906675 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd837a3a_0d04_4cb7_9812_b0f1e306a3b3.slice/crio-e2ac99766b404f6bdafab2136fa92e82d34179654c897fc9bfe884ce8cd49647 WatchSource:0}: Error finding container e2ac99766b404f6bdafab2136fa92e82d34179654c897fc9bfe884ce8cd49647: Status 404 returned error can't find the container with id e2ac99766b404f6bdafab2136fa92e82d34179654c897fc9bfe884ce8cd49647 Jul 22 16:13:43.043461 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:43.043357 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-994875d55-g2w9s" event={"ID":"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3","Type":"ContainerStarted","Data":"e2ac99766b404f6bdafab2136fa92e82d34179654c897fc9bfe884ce8cd49647"} Jul 22 16:13:44.005247 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.005209 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 16:13:44.010160 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.010134 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.016869 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.016622 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 22 16:13:44.016869 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.016640 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 22 16:13:44.016869 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.016653 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-12gpl98huo6n2\"" Jul 22 16:13:44.016869 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.016623 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 22 16:13:44.017703 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.017679 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 22 16:13:44.017808 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.017718 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 22 16:13:44.017808 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.017794 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 22 16:13:44.018068 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.017944 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bhszc\"" Jul 22 16:13:44.018068 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.018008 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 22 16:13:44.018233 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.018126 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 22 16:13:44.020165 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.020137 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 22 16:13:44.020577 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.020555 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 22 16:13:44.020829 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.020814 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 22 16:13:44.020920 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.020862 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 22 16:13:44.021022 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.021012 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 22 16:13:44.023244 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.023194 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 22 16:13:44.047345 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047310 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-web-config\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.047345 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047352 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.047345 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047370 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.047673 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047417 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.047673 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047443 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.047673 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047502 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.047673 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047544 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-config-out\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.047673 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047576 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.047673 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047622 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.047952 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047697 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.047952 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047722 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzxc9\" (UniqueName: \"kubernetes.io/projected/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-kube-api-access-lzxc9\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.047952 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047780 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.047952 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047822 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.047952 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047853 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.047952 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047884 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.047952 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047942 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.048290 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.047990 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-config\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.048290 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.048020 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.048290 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.048091 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.048290 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.048128 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.072420 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.072303 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 16:13:44.149335 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.149299 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-web-config\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.149335 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.149350 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.149600 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.149373 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.149600 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.149429 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.149600 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.149451 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.149600 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.149477 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.149600 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.149509 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-config-out\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.149600 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.149534 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.149600 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.149560 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.149945 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.149608 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.149945 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.149635 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzxc9\" (UniqueName: \"kubernetes.io/projected/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-kube-api-access-lzxc9\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.151317 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.150798 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.151317 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.150803 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.151317 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.150877 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.151317 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.150913 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.151317 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.150942 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.151317 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.150968 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.151317 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.151003 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-config\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.151317 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.151033 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.151317 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.151074 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.151317 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.151105 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.151317 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.151302 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.152545 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.151719 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.153103 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.152697 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.153103 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.152853 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.154002 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.153288 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.154002 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.153462 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.154002 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.153667 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.154341 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.154301 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.154341 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.154328 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.154823 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.154800 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-config-out\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.155048 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.155013 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-web-config\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.155870 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.155847 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.155965 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.155946 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.156906 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.156468 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.157278 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.157238 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-config\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.157647 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.157590 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.157954 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.157930 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.158463 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.158381 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.176261 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.176210 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzxc9\" (UniqueName: \"kubernetes.io/projected/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-kube-api-access-lzxc9\") pod \"prometheus-k8s-0\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.325478 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.325372 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:44.512362 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.512324 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 16:13:44.869321 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:13:44.869287 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26825dd9_17a9_4715_b9a9_99b6cd4f17a5.slice/crio-c8b3ba98d41d36a5c41611cd551131cdc60f48ad8e7080c26043d0b18bd1a42b WatchSource:0}: Error finding container c8b3ba98d41d36a5c41611cd551131cdc60f48ad8e7080c26043d0b18bd1a42b: Status 404 returned error can't find the container with id c8b3ba98d41d36a5c41611cd551131cdc60f48ad8e7080c26043d0b18bd1a42b Jul 22 16:13:44.942982 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:44.942958 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:13:45.052760 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:45.052721 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"26825dd9-17a9-4715-b9a9-99b6cd4f17a5","Type":"ContainerStarted","Data":"c8b3ba98d41d36a5c41611cd551131cdc60f48ad8e7080c26043d0b18bd1a42b"} Jul 22 16:13:45.054188 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:45.054166 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-994875d55-g2w9s" event={"ID":"dd837a3a-0d04-4cb7-9812-b0f1e306a3b3","Type":"ContainerStarted","Data":"36cdf1250dbb159491a3606e99cdc7d893a5deef7d3215b3ca1957b290f3ea4e"} Jul 22 16:13:46.059882 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:46.059852 2565 generic.go:356] "Generic (PLEG): container finished" podID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerID="5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01" exitCode=0 Jul 22 16:13:46.060181 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:46.059947 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"26825dd9-17a9-4715-b9a9-99b6cd4f17a5","Type":"ContainerDied","Data":"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01"} Jul 22 16:13:46.105492 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:46.105440 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-994875d55-g2w9s" podStartSLOduration=2.099612559 podStartE2EDuration="4.105424055s" totalImagesPullingTime="2.005811496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:45.096435441 +0000 UTC m=+122.109602479" watchObservedRunningTime="2026-07-22 16:13:46.105424055 +0000 UTC m=+123.118591094" Jul 22 16:13:48.936904 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:48.936864 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f8ffcd8f4-666kp"] Jul 22 16:13:50.080309 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:50.080273 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"26825dd9-17a9-4715-b9a9-99b6cd4f17a5","Type":"ContainerStarted","Data":"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09"} Jul 22 16:13:50.080723 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:50.080349 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"26825dd9-17a9-4715-b9a9-99b6cd4f17a5","Type":"ContainerStarted","Data":"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674"} Jul 22 16:13:52.093589 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:52.093553 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"26825dd9-17a9-4715-b9a9-99b6cd4f17a5","Type":"ContainerStarted","Data":"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6"} Jul 22 16:13:52.093962 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:52.093619 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"26825dd9-17a9-4715-b9a9-99b6cd4f17a5","Type":"ContainerStarted","Data":"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347"} Jul 22 16:13:52.093962 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:52.093645 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"26825dd9-17a9-4715-b9a9-99b6cd4f17a5","Type":"ContainerStarted","Data":"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c"} Jul 22 16:13:52.093962 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:52.093678 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"26825dd9-17a9-4715-b9a9-99b6cd4f17a5","Type":"ContainerStarted","Data":"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e"} Jul 22 16:13:52.133036 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:52.132987 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.901891397 podStartE2EDuration="9.132970385s" totalImagesPullingTime="6.211498955s" totalInitContainerRuntime="19.580033ms" isStatefulPod=false podCreationTimestamp="2026-07-22 16:13:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:52.130937974 +0000 UTC m=+129.144105014" watchObservedRunningTime="2026-07-22 16:13:52.132970385 +0000 UTC m=+129.146137424" Jul 22 16:13:53.337473 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:53.337433 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs\") pod \"network-metrics-daemon-gkw9w\" (UID: \"3dc7600c-4705-48d3-a410-eebfaa40c9cf\") " pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:13:53.340556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:53.340528 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3dc7600c-4705-48d3-a410-eebfaa40c9cf-metrics-certs\") pod \"network-metrics-daemon-gkw9w\" (UID: \"3dc7600c-4705-48d3-a410-eebfaa40c9cf\") " pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:13:53.537727 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:53.537701 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-df68t\"" Jul 22 16:13:53.545875 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:53.545601 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gkw9w" Jul 22 16:13:53.696700 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:53.696676 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gkw9w"] Jul 22 16:13:53.700097 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:13:53.700062 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dc7600c_4705_48d3_a410_eebfaa40c9cf.slice/crio-fb523b61014026740a9ca5e3990089e9c4955bfce5617a1a9ab727cfcf9aaa5f WatchSource:0}: Error finding container fb523b61014026740a9ca5e3990089e9c4955bfce5617a1a9ab727cfcf9aaa5f: Status 404 returned error can't find the container with id fb523b61014026740a9ca5e3990089e9c4955bfce5617a1a9ab727cfcf9aaa5f Jul 22 16:13:54.103643 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:54.103609 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gkw9w" event={"ID":"3dc7600c-4705-48d3-a410-eebfaa40c9cf","Type":"ContainerStarted","Data":"fb523b61014026740a9ca5e3990089e9c4955bfce5617a1a9ab727cfcf9aaa5f"} Jul 22 16:13:54.326551 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:54.326523 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:13:55.109540 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:55.109468 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gkw9w" event={"ID":"3dc7600c-4705-48d3-a410-eebfaa40c9cf","Type":"ContainerStarted","Data":"4cbd785672a073297f3e822ace13ddc118d1a5d3490a893144663b5e6efec9f6"} Jul 22 16:13:55.109857 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:55.109558 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gkw9w" event={"ID":"3dc7600c-4705-48d3-a410-eebfaa40c9cf","Type":"ContainerStarted","Data":"e435ec91ac4fa394d20eacbf377b6667c5805465fec16596b34bd76e257e89fa"} Jul 22 16:13:55.137095 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:13:55.137045 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gkw9w" podStartSLOduration=129.982362713 podStartE2EDuration="2m11.13703096s" totalImagesPullingTime="1.154668247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:13:55.136336486 +0000 UTC m=+132.149503525" watchObservedRunningTime="2026-07-22 16:13:55.13703096 +0000 UTC m=+132.150197998" Jul 22 16:14:02.770594 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:02.770560 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:14:02.771037 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:02.770617 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:14:13.960820 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:13.960754 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" podUID="26e2f0f2-77f1-45d0-88b5-bb10285392d0" containerName="registry" containerID="cri-o://d57a5c405d09431b38def6f4d5e18ade0fc69280df2faf0b3a4e443a6e2af7af" gracePeriod=30 Jul 22 16:14:14.181308 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.181280 2565 generic.go:356] "Generic (PLEG): container finished" podID="26e2f0f2-77f1-45d0-88b5-bb10285392d0" containerID="d57a5c405d09431b38def6f4d5e18ade0fc69280df2faf0b3a4e443a6e2af7af" exitCode=0 Jul 22 16:14:14.181466 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.181355 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" event={"ID":"26e2f0f2-77f1-45d0-88b5-bb10285392d0","Type":"ContainerDied","Data":"d57a5c405d09431b38def6f4d5e18ade0fc69280df2faf0b3a4e443a6e2af7af"} Jul 22 16:14:14.216626 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.216575 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:14:14.297368 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.297338 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/26e2f0f2-77f1-45d0-88b5-bb10285392d0-ca-trust-extracted\") pod \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " Jul 22 16:14:14.297520 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.297401 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/26e2f0f2-77f1-45d0-88b5-bb10285392d0-installation-pull-secrets\") pod \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " Jul 22 16:14:14.297520 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.297481 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls\") pod \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " Jul 22 16:14:14.297520 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.297510 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-certificates\") pod \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " Jul 22 16:14:14.297679 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.297538 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8grf4\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-kube-api-access-8grf4\") pod \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " Jul 22 16:14:14.297679 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.297572 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/26e2f0f2-77f1-45d0-88b5-bb10285392d0-trusted-ca\") pod \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " Jul 22 16:14:14.297679 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.297611 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/26e2f0f2-77f1-45d0-88b5-bb10285392d0-image-registry-private-configuration\") pod \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " Jul 22 16:14:14.297679 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.297641 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-bound-sa-token\") pod \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\" (UID: \"26e2f0f2-77f1-45d0-88b5-bb10285392d0\") " Jul 22 16:14:14.298072 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.298015 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "26e2f0f2-77f1-45d0-88b5-bb10285392d0" (UID: "26e2f0f2-77f1-45d0-88b5-bb10285392d0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:14:14.298191 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.298103 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26e2f0f2-77f1-45d0-88b5-bb10285392d0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "26e2f0f2-77f1-45d0-88b5-bb10285392d0" (UID: "26e2f0f2-77f1-45d0-88b5-bb10285392d0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:14:14.300715 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.300680 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-kube-api-access-8grf4" (OuterVolumeSpecName: "kube-api-access-8grf4") pod "26e2f0f2-77f1-45d0-88b5-bb10285392d0" (UID: "26e2f0f2-77f1-45d0-88b5-bb10285392d0"). InnerVolumeSpecName "kube-api-access-8grf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:14:14.300812 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.300689 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26e2f0f2-77f1-45d0-88b5-bb10285392d0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "26e2f0f2-77f1-45d0-88b5-bb10285392d0" (UID: "26e2f0f2-77f1-45d0-88b5-bb10285392d0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:14:14.300812 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.300750 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "26e2f0f2-77f1-45d0-88b5-bb10285392d0" (UID: "26e2f0f2-77f1-45d0-88b5-bb10285392d0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:14:14.300812 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.300755 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "26e2f0f2-77f1-45d0-88b5-bb10285392d0" (UID: "26e2f0f2-77f1-45d0-88b5-bb10285392d0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:14:14.300989 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.300964 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26e2f0f2-77f1-45d0-88b5-bb10285392d0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "26e2f0f2-77f1-45d0-88b5-bb10285392d0" (UID: "26e2f0f2-77f1-45d0-88b5-bb10285392d0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:14:14.305838 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.305816 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26e2f0f2-77f1-45d0-88b5-bb10285392d0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "26e2f0f2-77f1-45d0-88b5-bb10285392d0" (UID: "26e2f0f2-77f1-45d0-88b5-bb10285392d0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:14:14.398748 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.398729 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:14:14.398834 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.398750 2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/26e2f0f2-77f1-45d0-88b5-bb10285392d0-registry-certificates\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:14:14.398834 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.398761 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8grf4\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-kube-api-access-8grf4\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:14:14.398834 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.398770 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/26e2f0f2-77f1-45d0-88b5-bb10285392d0-trusted-ca\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:14:14.398834 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.398783 2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/26e2f0f2-77f1-45d0-88b5-bb10285392d0-image-registry-private-configuration\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:14:14.398834 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.398798 2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/26e2f0f2-77f1-45d0-88b5-bb10285392d0-bound-sa-token\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:14:14.398834 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.398807 2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/26e2f0f2-77f1-45d0-88b5-bb10285392d0-ca-trust-extracted\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:14:14.398834 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:14.398814 2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/26e2f0f2-77f1-45d0-88b5-bb10285392d0-installation-pull-secrets\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:14:15.187083 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:15.187055 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" Jul 22 16:14:15.187523 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:15.187050 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f8ffcd8f4-666kp" event={"ID":"26e2f0f2-77f1-45d0-88b5-bb10285392d0","Type":"ContainerDied","Data":"0c34af9022e9a675a0b8c997d21379e950eec659d3e541e26fe6276a937415f5"} Jul 22 16:14:15.187523 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:15.187192 2565 scope.go:117] "RemoveContainer" containerID="d57a5c405d09431b38def6f4d5e18ade0fc69280df2faf0b3a4e443a6e2af7af" Jul 22 16:14:15.217057 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:15.217027 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f8ffcd8f4-666kp"] Jul 22 16:14:15.228446 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:15.228423 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5f8ffcd8f4-666kp"] Jul 22 16:14:15.526958 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:15.526902 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26e2f0f2-77f1-45d0-88b5-bb10285392d0" path="/var/lib/kubelet/pods/26e2f0f2-77f1-45d0-88b5-bb10285392d0/volumes" Jul 22 16:14:22.775720 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:22.775693 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:14:22.779640 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:22.779618 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-994875d55-g2w9s" Jul 22 16:14:25.230720 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:25.230674 2565 generic.go:356] "Generic (PLEG): container finished" podID="02ac6317-0f6c-4fc4-81df-699216b0f136" containerID="f484d6a63bfe05b480fd277adcd3c17167eea288746a23f3882b93f4306d43ac" exitCode=0 Jul 22 16:14:25.231188 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:25.230756 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6" event={"ID":"02ac6317-0f6c-4fc4-81df-699216b0f136","Type":"ContainerDied","Data":"f484d6a63bfe05b480fd277adcd3c17167eea288746a23f3882b93f4306d43ac"} Jul 22 16:14:25.231188 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:25.231179 2565 scope.go:117] "RemoveContainer" containerID="f484d6a63bfe05b480fd277adcd3c17167eea288746a23f3882b93f4306d43ac" Jul 22 16:14:25.232790 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:25.232663 2565 generic.go:356] "Generic (PLEG): container finished" podID="592cc5de-ba98-448f-9120-4690a117a4ca" containerID="2d8da70ae1dc53146aa1ebf87bfd4afdbc5e43bde4fe7f780e3dd5e2a9ab067a" exitCode=0 Jul 22 16:14:25.232790 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:25.232690 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f" event={"ID":"592cc5de-ba98-448f-9120-4690a117a4ca","Type":"ContainerDied","Data":"2d8da70ae1dc53146aa1ebf87bfd4afdbc5e43bde4fe7f780e3dd5e2a9ab067a"} Jul 22 16:14:25.233004 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:25.232986 2565 scope.go:117] "RemoveContainer" containerID="2d8da70ae1dc53146aa1ebf87bfd4afdbc5e43bde4fe7f780e3dd5e2a9ab067a" Jul 22 16:14:26.241596 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:26.241555 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nzb4f" event={"ID":"592cc5de-ba98-448f-9120-4690a117a4ca","Type":"ContainerStarted","Data":"9bd5d53081fd0ec39e4e366a1771ccca45bd846c897005ec6ab2948b5c2daa46"} Jul 22 16:14:26.245145 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:26.245107 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-6ckq6" event={"ID":"02ac6317-0f6c-4fc4-81df-699216b0f136","Type":"ContainerStarted","Data":"02634c325603aed5e366372a6d4f5cd64fe6cd0f96c37ded930f544c8b10836d"} Jul 22 16:14:44.326113 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:44.326076 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:14:44.345370 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:44.345345 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:14:45.346329 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:14:45.346289 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:02.651477 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.651432 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 16:15:02.652008 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.651727 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="prometheus" containerID="cri-o://a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674" gracePeriod=600 Jul 22 16:15:02.652008 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.651766 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="kube-rbac-proxy" containerID="cri-o://4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347" gracePeriod=600 Jul 22 16:15:02.652008 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.651797 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6" gracePeriod=600 Jul 22 16:15:02.652008 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.651823 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="thanos-sidecar" containerID="cri-o://83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e" gracePeriod=600 Jul 22 16:15:02.652008 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.651871 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="kube-rbac-proxy-web" containerID="cri-o://58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c" gracePeriod=600 Jul 22 16:15:02.652008 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.651827 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="config-reloader" containerID="cri-o://96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09" gracePeriod=600 Jul 22 16:15:02.903291 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.903217 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:02.972515 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972204 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-config-out\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.972515 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972249 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-2\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.972515 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972285 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.972515 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972316 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-tls-assets\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.972515 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972347 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-0\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.972515 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972381 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-trusted-ca-bundle\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.972515 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972444 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-serving-certs-ca-bundle\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.972515 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972470 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-thanos-prometheus-http-client-file\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.972515 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972504 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-db\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.973048 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972529 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-kubelet-serving-ca-bundle\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.973048 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972550 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-tls\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.973048 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972588 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.973048 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972615 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-config\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.973048 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972638 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-metrics-client-certs\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.973048 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972663 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzxc9\" (UniqueName: \"kubernetes.io/projected/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-kube-api-access-lzxc9\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.973048 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972692 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-metrics-client-ca\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.973048 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972714 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-kube-rbac-proxy\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.973048 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972740 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-grpc-tls\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.973048 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972777 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-web-config\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.973048 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.972809 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-1\") pod \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\" (UID: \"26825dd9-17a9-4715-b9a9-99b6cd4f17a5\") " Jul 22 16:15:02.974741 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.974549 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:15:02.974741 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.974577 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:15:02.974929 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.974802 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:15:02.975746 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.974970 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:15:02.975746 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.975054 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:15:02.975746 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.975295 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:15:02.975746 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.975446 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:15:02.976020 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.976002 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:15:02.979609 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.979583 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:15:02.981184 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.981161 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:15:02.981493 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.981463 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:15:02.981733 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.981710 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-config" (OuterVolumeSpecName: "config") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:15:02.981972 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.981933 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-kube-api-access-lzxc9" (OuterVolumeSpecName: "kube-api-access-lzxc9") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "kube-api-access-lzxc9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:15:02.982071 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.982047 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:15:02.982131 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.982097 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:15:02.982187 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.982164 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-config-out" (OuterVolumeSpecName: "config-out") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:15:02.982497 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.982373 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:15:02.982497 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.982443 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:15:02.982497 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.982473 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:15:02.995382 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:02.995338 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-web-config" (OuterVolumeSpecName: "web-config") pod "26825dd9-17a9-4715-b9a9-99b6cd4f17a5" (UID: "26825dd9-17a9-4715-b9a9-99b6cd4f17a5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:15:03.073877 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.073830 2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-config-out\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.073877 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.073873 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.073877 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.073885 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.073877 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.073894 2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-tls-assets\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.074129 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.073903 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.074129 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.073911 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.074129 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.073921 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.074129 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.073929 2565 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.074129 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.073937 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-db\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.074129 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.073946 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.074129 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.073954 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.074129 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.073962 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.074129 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.073970 2565 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-config\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.074129 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.073978 2565 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-metrics-client-certs\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.074129 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.073987 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzxc9\" (UniqueName: \"kubernetes.io/projected/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-kube-api-access-lzxc9\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.074129 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.073994 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-configmap-metrics-client-ca\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.074129 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.074002 2565 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-kube-rbac-proxy\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.074129 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.074011 2565 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-secret-grpc-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.074129 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.074018 2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-web-config\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.074129 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.074025 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/26825dd9-17a9-4715-b9a9-99b6cd4f17a5-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:15:03.403227 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.403190 2565 generic.go:356] "Generic (PLEG): container finished" podID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerID="d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6" exitCode=0 Jul 22 16:15:03.403227 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.403217 2565 generic.go:356] "Generic (PLEG): container finished" podID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerID="4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347" exitCode=0 Jul 22 16:15:03.403227 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.403223 2565 generic.go:356] "Generic (PLEG): container finished" podID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerID="58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c" exitCode=0 Jul 22 16:15:03.403227 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.403229 2565 generic.go:356] "Generic (PLEG): container finished" podID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerID="83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e" exitCode=0 Jul 22 16:15:03.403227 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.403234 2565 generic.go:356] "Generic (PLEG): container finished" podID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerID="96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09" exitCode=0 Jul 22 16:15:03.403227 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.403239 2565 generic.go:356] "Generic (PLEG): container finished" podID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerID="a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674" exitCode=0 Jul 22 16:15:03.403689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.403260 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"26825dd9-17a9-4715-b9a9-99b6cd4f17a5","Type":"ContainerDied","Data":"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6"} Jul 22 16:15:03.403689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.403290 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.403689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.403358 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"26825dd9-17a9-4715-b9a9-99b6cd4f17a5","Type":"ContainerDied","Data":"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347"} Jul 22 16:15:03.403689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.403402 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"26825dd9-17a9-4715-b9a9-99b6cd4f17a5","Type":"ContainerDied","Data":"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c"} Jul 22 16:15:03.403689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.403414 2565 scope.go:117] "RemoveContainer" containerID="d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6" Jul 22 16:15:03.403689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.403428 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"26825dd9-17a9-4715-b9a9-99b6cd4f17a5","Type":"ContainerDied","Data":"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e"} Jul 22 16:15:03.403689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.403452 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"26825dd9-17a9-4715-b9a9-99b6cd4f17a5","Type":"ContainerDied","Data":"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09"} Jul 22 16:15:03.403689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.403475 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"26825dd9-17a9-4715-b9a9-99b6cd4f17a5","Type":"ContainerDied","Data":"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674"} Jul 22 16:15:03.403689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.403499 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"26825dd9-17a9-4715-b9a9-99b6cd4f17a5","Type":"ContainerDied","Data":"c8b3ba98d41d36a5c41611cd551131cdc60f48ad8e7080c26043d0b18bd1a42b"} Jul 22 16:15:03.411729 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.411702 2565 scope.go:117] "RemoveContainer" containerID="4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347" Jul 22 16:15:03.418482 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.418416 2565 scope.go:117] "RemoveContainer" containerID="58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c" Jul 22 16:15:03.425118 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.425102 2565 scope.go:117] "RemoveContainer" containerID="83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e" Jul 22 16:15:03.431548 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.431530 2565 scope.go:117] "RemoveContainer" containerID="96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09" Jul 22 16:15:03.435055 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.435032 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 16:15:03.438632 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.438613 2565 scope.go:117] "RemoveContainer" containerID="a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674" Jul 22 16:15:03.445101 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.445079 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 16:15:03.446231 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.446216 2565 scope.go:117] "RemoveContainer" containerID="5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01" Jul 22 16:15:03.452718 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.452702 2565 scope.go:117] "RemoveContainer" containerID="d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6" Jul 22 16:15:03.453013 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:15:03.452989 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6\": container with ID starting with d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6 not found: ID does not exist" containerID="d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6" Jul 22 16:15:03.453067 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.453024 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6"} err="failed to get container status \"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6\": rpc error: code = NotFound desc = could not find container \"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6\": container with ID starting with d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6 not found: ID does not exist" Jul 22 16:15:03.453067 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.453058 2565 scope.go:117] "RemoveContainer" containerID="4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347" Jul 22 16:15:03.453296 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:15:03.453279 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347\": container with ID starting with 4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347 not found: ID does not exist" containerID="4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347" Jul 22 16:15:03.453332 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.453303 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347"} err="failed to get container status \"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347\": rpc error: code = NotFound desc = could not find container \"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347\": container with ID starting with 4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347 not found: ID does not exist" Jul 22 16:15:03.453332 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.453319 2565 scope.go:117] "RemoveContainer" containerID="58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c" Jul 22 16:15:03.453537 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:15:03.453513 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c\": container with ID starting with 58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c not found: ID does not exist" containerID="58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c" Jul 22 16:15:03.453585 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.453543 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c"} err="failed to get container status \"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c\": rpc error: code = NotFound desc = could not find container \"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c\": container with ID starting with 58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c not found: ID does not exist" Jul 22 16:15:03.453585 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.453557 2565 scope.go:117] "RemoveContainer" containerID="83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e" Jul 22 16:15:03.453818 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:15:03.453796 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e\": container with ID starting with 83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e not found: ID does not exist" containerID="83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e" Jul 22 16:15:03.453858 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.453820 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e"} err="failed to get container status \"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e\": rpc error: code = NotFound desc = could not find container \"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e\": container with ID starting with 83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e not found: ID does not exist" Jul 22 16:15:03.453858 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.453834 2565 scope.go:117] "RemoveContainer" containerID="96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09" Jul 22 16:15:03.454072 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:15:03.454050 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09\": container with ID starting with 96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09 not found: ID does not exist" containerID="96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09" Jul 22 16:15:03.454127 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.454077 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09"} err="failed to get container status \"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09\": rpc error: code = NotFound desc = could not find container \"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09\": container with ID starting with 96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09 not found: ID does not exist" Jul 22 16:15:03.454127 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.454093 2565 scope.go:117] "RemoveContainer" containerID="a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674" Jul 22 16:15:03.454299 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:15:03.454281 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674\": container with ID starting with a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674 not found: ID does not exist" containerID="a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674" Jul 22 16:15:03.454362 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.454307 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674"} err="failed to get container status \"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674\": rpc error: code = NotFound desc = could not find container \"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674\": container with ID starting with a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674 not found: ID does not exist" Jul 22 16:15:03.454362 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.454327 2565 scope.go:117] "RemoveContainer" containerID="5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01" Jul 22 16:15:03.454584 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:15:03.454564 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01\": container with ID starting with 5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01 not found: ID does not exist" containerID="5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01" Jul 22 16:15:03.454682 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.454586 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01"} err="failed to get container status \"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01\": rpc error: code = NotFound desc = could not find container \"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01\": container with ID starting with 5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01 not found: ID does not exist" Jul 22 16:15:03.454682 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.454601 2565 scope.go:117] "RemoveContainer" containerID="d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6" Jul 22 16:15:03.454835 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.454817 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6"} err="failed to get container status \"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6\": rpc error: code = NotFound desc = could not find container \"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6\": container with ID starting with d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6 not found: ID does not exist" Jul 22 16:15:03.454878 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.454836 2565 scope.go:117] "RemoveContainer" containerID="4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347" Jul 22 16:15:03.455080 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.455058 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347"} err="failed to get container status \"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347\": rpc error: code = NotFound desc = could not find container \"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347\": container with ID starting with 4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347 not found: ID does not exist" Jul 22 16:15:03.455080 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.455080 2565 scope.go:117] "RemoveContainer" containerID="58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c" Jul 22 16:15:03.455308 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.455284 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c"} err="failed to get container status \"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c\": rpc error: code = NotFound desc = could not find container \"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c\": container with ID starting with 58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c not found: ID does not exist" Jul 22 16:15:03.455308 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.455308 2565 scope.go:117] "RemoveContainer" containerID="83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e" Jul 22 16:15:03.455550 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.455533 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e"} err="failed to get container status \"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e\": rpc error: code = NotFound desc = could not find container \"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e\": container with ID starting with 83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e not found: ID does not exist" Jul 22 16:15:03.455604 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.455551 2565 scope.go:117] "RemoveContainer" containerID="96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09" Jul 22 16:15:03.455778 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.455760 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09"} err="failed to get container status \"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09\": rpc error: code = NotFound desc = could not find container \"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09\": container with ID starting with 96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09 not found: ID does not exist" Jul 22 16:15:03.455778 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.455777 2565 scope.go:117] "RemoveContainer" containerID="a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674" Jul 22 16:15:03.456003 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.455981 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674"} err="failed to get container status \"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674\": rpc error: code = NotFound desc = could not find container \"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674\": container with ID starting with a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674 not found: ID does not exist" Jul 22 16:15:03.456049 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.456005 2565 scope.go:117] "RemoveContainer" containerID="5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01" Jul 22 16:15:03.456213 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.456197 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01"} err="failed to get container status \"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01\": rpc error: code = NotFound desc = could not find container \"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01\": container with ID starting with 5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01 not found: ID does not exist" Jul 22 16:15:03.456256 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.456214 2565 scope.go:117] "RemoveContainer" containerID="d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6" Jul 22 16:15:03.456427 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.456407 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6"} err="failed to get container status \"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6\": rpc error: code = NotFound desc = could not find container \"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6\": container with ID starting with d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6 not found: ID does not exist" Jul 22 16:15:03.456474 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.456427 2565 scope.go:117] "RemoveContainer" containerID="4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347" Jul 22 16:15:03.456620 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.456599 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347"} err="failed to get container status \"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347\": rpc error: code = NotFound desc = could not find container \"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347\": container with ID starting with 4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347 not found: ID does not exist" Jul 22 16:15:03.456620 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.456618 2565 scope.go:117] "RemoveContainer" containerID="58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c" Jul 22 16:15:03.456813 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.456798 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c"} err="failed to get container status \"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c\": rpc error: code = NotFound desc = could not find container \"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c\": container with ID starting with 58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c not found: ID does not exist" Jul 22 16:15:03.456857 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.456814 2565 scope.go:117] "RemoveContainer" containerID="83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e" Jul 22 16:15:03.457024 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.457008 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e"} err="failed to get container status \"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e\": rpc error: code = NotFound desc = could not find container \"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e\": container with ID starting with 83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e not found: ID does not exist" Jul 22 16:15:03.457063 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.457026 2565 scope.go:117] "RemoveContainer" containerID="96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09" Jul 22 16:15:03.457217 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.457203 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09"} err="failed to get container status \"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09\": rpc error: code = NotFound desc = could not find container \"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09\": container with ID starting with 96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09 not found: ID does not exist" Jul 22 16:15:03.457264 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.457217 2565 scope.go:117] "RemoveContainer" containerID="a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674" Jul 22 16:15:03.457438 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.457423 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674"} err="failed to get container status \"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674\": rpc error: code = NotFound desc = could not find container \"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674\": container with ID starting with a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674 not found: ID does not exist" Jul 22 16:15:03.457497 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.457438 2565 scope.go:117] "RemoveContainer" containerID="5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01" Jul 22 16:15:03.457623 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.457604 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01"} err="failed to get container status \"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01\": rpc error: code = NotFound desc = could not find container \"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01\": container with ID starting with 5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01 not found: ID does not exist" Jul 22 16:15:03.457673 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.457624 2565 scope.go:117] "RemoveContainer" containerID="d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6" Jul 22 16:15:03.457865 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.457844 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6"} err="failed to get container status \"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6\": rpc error: code = NotFound desc = could not find container \"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6\": container with ID starting with d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6 not found: ID does not exist" Jul 22 16:15:03.457865 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.457865 2565 scope.go:117] "RemoveContainer" containerID="4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347" Jul 22 16:15:03.458071 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.458055 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347"} err="failed to get container status \"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347\": rpc error: code = NotFound desc = could not find container \"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347\": container with ID starting with 4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347 not found: ID does not exist" Jul 22 16:15:03.458115 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.458071 2565 scope.go:117] "RemoveContainer" containerID="58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c" Jul 22 16:15:03.458251 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.458234 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c"} err="failed to get container status \"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c\": rpc error: code = NotFound desc = could not find container \"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c\": container with ID starting with 58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c not found: ID does not exist" Jul 22 16:15:03.458251 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.458251 2565 scope.go:117] "RemoveContainer" containerID="83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e" Jul 22 16:15:03.458519 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.458501 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e"} err="failed to get container status \"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e\": rpc error: code = NotFound desc = could not find container \"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e\": container with ID starting with 83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e not found: ID does not exist" Jul 22 16:15:03.458580 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.458521 2565 scope.go:117] "RemoveContainer" containerID="96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09" Jul 22 16:15:03.458739 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.458722 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09"} err="failed to get container status \"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09\": rpc error: code = NotFound desc = could not find container \"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09\": container with ID starting with 96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09 not found: ID does not exist" Jul 22 16:15:03.458779 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.458741 2565 scope.go:117] "RemoveContainer" containerID="a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674" Jul 22 16:15:03.458963 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.458943 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674"} err="failed to get container status \"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674\": rpc error: code = NotFound desc = could not find container \"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674\": container with ID starting with a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674 not found: ID does not exist" Jul 22 16:15:03.459004 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.458966 2565 scope.go:117] "RemoveContainer" containerID="5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01" Jul 22 16:15:03.459170 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.459155 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01"} err="failed to get container status \"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01\": rpc error: code = NotFound desc = could not find container \"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01\": container with ID starting with 5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01 not found: ID does not exist" Jul 22 16:15:03.459170 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.459169 2565 scope.go:117] "RemoveContainer" containerID="d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6" Jul 22 16:15:03.459434 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.459383 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6"} err="failed to get container status \"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6\": rpc error: code = NotFound desc = could not find container \"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6\": container with ID starting with d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6 not found: ID does not exist" Jul 22 16:15:03.459434 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.459421 2565 scope.go:117] "RemoveContainer" containerID="4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347" Jul 22 16:15:03.459614 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.459592 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347"} err="failed to get container status \"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347\": rpc error: code = NotFound desc = could not find container \"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347\": container with ID starting with 4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347 not found: ID does not exist" Jul 22 16:15:03.459656 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.459617 2565 scope.go:117] "RemoveContainer" containerID="58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c" Jul 22 16:15:03.459798 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.459779 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c"} err="failed to get container status \"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c\": rpc error: code = NotFound desc = could not find container \"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c\": container with ID starting with 58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c not found: ID does not exist" Jul 22 16:15:03.459798 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.459799 2565 scope.go:117] "RemoveContainer" containerID="83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e" Jul 22 16:15:03.460022 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.460001 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e"} err="failed to get container status \"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e\": rpc error: code = NotFound desc = could not find container \"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e\": container with ID starting with 83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e not found: ID does not exist" Jul 22 16:15:03.460091 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.460023 2565 scope.go:117] "RemoveContainer" containerID="96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09" Jul 22 16:15:03.460230 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.460214 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09"} err="failed to get container status \"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09\": rpc error: code = NotFound desc = could not find container \"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09\": container with ID starting with 96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09 not found: ID does not exist" Jul 22 16:15:03.460300 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.460232 2565 scope.go:117] "RemoveContainer" containerID="a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674" Jul 22 16:15:03.460490 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.460465 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674"} err="failed to get container status \"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674\": rpc error: code = NotFound desc = could not find container \"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674\": container with ID starting with a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674 not found: ID does not exist" Jul 22 16:15:03.460490 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.460482 2565 scope.go:117] "RemoveContainer" containerID="5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01" Jul 22 16:15:03.460681 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.460664 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01"} err="failed to get container status \"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01\": rpc error: code = NotFound desc = could not find container \"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01\": container with ID starting with 5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01 not found: ID does not exist" Jul 22 16:15:03.460731 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.460682 2565 scope.go:117] "RemoveContainer" containerID="d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6" Jul 22 16:15:03.460898 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.460882 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6"} err="failed to get container status \"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6\": rpc error: code = NotFound desc = could not find container \"d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6\": container with ID starting with d621dbedad3f425536399a292fae130ebe537bf1e547c5d705fa9097881b46f6 not found: ID does not exist" Jul 22 16:15:03.460950 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.460899 2565 scope.go:117] "RemoveContainer" containerID="4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347" Jul 22 16:15:03.461067 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.461051 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347"} err="failed to get container status \"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347\": rpc error: code = NotFound desc = could not find container \"4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347\": container with ID starting with 4d647faf9f38d7b6a7414dceefc239a1e6a142b807cc5f922f1fd5949844c347 not found: ID does not exist" Jul 22 16:15:03.461107 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.461067 2565 scope.go:117] "RemoveContainer" containerID="58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c" Jul 22 16:15:03.461269 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.461255 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c"} err="failed to get container status \"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c\": rpc error: code = NotFound desc = could not find container \"58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c\": container with ID starting with 58e62b2f64d40e25c9370d03b57264b2eb217e371f86e311e97c3a34cddd635c not found: ID does not exist" Jul 22 16:15:03.461305 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.461269 2565 scope.go:117] "RemoveContainer" containerID="83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e" Jul 22 16:15:03.461541 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.461523 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e"} err="failed to get container status \"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e\": rpc error: code = NotFound desc = could not find container \"83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e\": container with ID starting with 83d6f16937c85728451de955aaa610ece7bba4c642b65829d4b7da5a58e0a07e not found: ID does not exist" Jul 22 16:15:03.461613 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.461543 2565 scope.go:117] "RemoveContainer" containerID="96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09" Jul 22 16:15:03.461769 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.461754 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09"} err="failed to get container status \"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09\": rpc error: code = NotFound desc = could not find container \"96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09\": container with ID starting with 96fb435474dc715c8c6cb8027439a0f61475b6a5672c2edd8e5319e0b7532d09 not found: ID does not exist" Jul 22 16:15:03.461769 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.461768 2565 scope.go:117] "RemoveContainer" containerID="a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674" Jul 22 16:15:03.461971 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.461956 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674"} err="failed to get container status \"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674\": rpc error: code = NotFound desc = could not find container \"a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674\": container with ID starting with a8b4c7312eebba2883c16e97560e40b2169e87731f61e4e8009e735858f32674 not found: ID does not exist" Jul 22 16:15:03.462010 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.461972 2565 scope.go:117] "RemoveContainer" containerID="5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01" Jul 22 16:15:03.462150 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.462135 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01"} err="failed to get container status \"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01\": rpc error: code = NotFound desc = could not find container \"5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01\": container with ID starting with 5cabf99adb00b31576056a83f5b8ddd2fe47ccad14d71d3f762490378127eb01 not found: ID does not exist" Jul 22 16:15:03.491217 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491191 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 16:15:03.491561 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491543 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="kube-rbac-proxy-web" Jul 22 16:15:03.491651 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491562 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="kube-rbac-proxy-web" Jul 22 16:15:03.491651 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491580 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26e2f0f2-77f1-45d0-88b5-bb10285392d0" containerName="registry" Jul 22 16:15:03.491651 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491590 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="26e2f0f2-77f1-45d0-88b5-bb10285392d0" containerName="registry" Jul 22 16:15:03.491651 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491603 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="thanos-sidecar" Jul 22 16:15:03.491651 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491613 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="thanos-sidecar" Jul 22 16:15:03.491651 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491626 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="init-config-reloader" Jul 22 16:15:03.491651 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491635 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="init-config-reloader" Jul 22 16:15:03.491651 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491656 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="prometheus" Jul 22 16:15:03.491949 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491662 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="prometheus" Jul 22 16:15:03.491949 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491668 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="config-reloader" Jul 22 16:15:03.491949 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491676 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="config-reloader" Jul 22 16:15:03.491949 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491685 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="kube-rbac-proxy" Jul 22 16:15:03.491949 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491693 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="kube-rbac-proxy" Jul 22 16:15:03.491949 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491701 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="kube-rbac-proxy-thanos" Jul 22 16:15:03.491949 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491710 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="kube-rbac-proxy-thanos" Jul 22 16:15:03.491949 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491776 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="config-reloader" Jul 22 16:15:03.491949 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491787 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="thanos-sidecar" Jul 22 16:15:03.491949 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491795 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="kube-rbac-proxy" Jul 22 16:15:03.491949 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491804 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="kube-rbac-proxy-thanos" Jul 22 16:15:03.491949 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491815 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="26e2f0f2-77f1-45d0-88b5-bb10285392d0" containerName="registry" Jul 22 16:15:03.491949 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491824 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="prometheus" Jul 22 16:15:03.491949 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.491836 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" containerName="kube-rbac-proxy-web" Jul 22 16:15:03.495214 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.495199 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.501731 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.501712 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 22 16:15:03.501977 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.501960 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 22 16:15:03.504500 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.504478 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 22 16:15:03.504691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.504670 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 22 16:15:03.504784 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.504745 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 22 16:15:03.505094 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.505068 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 22 16:15:03.505185 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.505116 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 22 16:15:03.505245 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.505187 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 22 16:15:03.505471 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.505433 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 22 16:15:03.505471 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.505450 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bhszc\"" Jul 22 16:15:03.505618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.505508 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 22 16:15:03.505618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.505443 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 22 16:15:03.505618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.505425 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-12gpl98huo6n2\"" Jul 22 16:15:03.506473 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.506458 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 22 16:15:03.508223 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.508207 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 22 16:15:03.509505 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.509491 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 22 16:15:03.530065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.528094 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26825dd9-17a9-4715-b9a9-99b6cd4f17a5" path="/var/lib/kubelet/pods/26825dd9-17a9-4715-b9a9-99b6cd4f17a5/volumes" Jul 22 16:15:03.530464 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.530441 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 16:15:03.578478 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578444 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.578478 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578483 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-web-config\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.578616 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578501 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6f5cc774-8c4e-43fc-a268-8b99de6e102e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.578616 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578532 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.578616 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578555 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.578616 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578573 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.578748 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578645 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.578748 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578664 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.578748 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578710 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f5cc774-8c4e-43fc-a268-8b99de6e102e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.578748 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578730 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.578867 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578775 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.578867 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578796 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.578867 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578822 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.578867 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578844 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9ft7\" (UniqueName: \"kubernetes.io/projected/6f5cc774-8c4e-43fc-a268-8b99de6e102e-kube-api-access-k9ft7\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.578977 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578865 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.578977 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578882 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.578977 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578945 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-config\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.579061 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.578986 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.579061 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.579008 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f5cc774-8c4e-43fc-a268-8b99de6e102e-config-out\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.579061 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.579022 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.679731 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.679703 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f5cc774-8c4e-43fc-a268-8b99de6e102e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680090 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.679738 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680090 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.679764 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680090 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.679790 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680090 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.679820 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680090 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.679923 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9ft7\" (UniqueName: \"kubernetes.io/projected/6f5cc774-8c4e-43fc-a268-8b99de6e102e-kube-api-access-k9ft7\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680090 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.679965 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680090 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.679993 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680090 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.680029 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-config\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680090 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.680054 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680090 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.680081 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f5cc774-8c4e-43fc-a268-8b99de6e102e-config-out\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680592 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.680108 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680592 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.680144 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680592 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.680174 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-web-config\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680592 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.680198 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6f5cc774-8c4e-43fc-a268-8b99de6e102e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680592 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.680243 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680592 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.680268 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680592 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.680306 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680592 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.680345 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680592 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.680369 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680994 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.680607 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.680994 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.680772 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.681094 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.681074 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.681151 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.681097 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.681541 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.681516 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.681713 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.681540 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.683257 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.682620 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f5cc774-8c4e-43fc-a268-8b99de6e102e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.684819 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.684768 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6f5cc774-8c4e-43fc-a268-8b99de6e102e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.685152 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.684875 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.685152 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.684978 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.685152 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.685000 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.685313 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.685263 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.685313 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.685296 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.685591 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.685570 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f5cc774-8c4e-43fc-a268-8b99de6e102e-config-out\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.685798 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.685765 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.686134 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.686113 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-web-config\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.686425 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.686380 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.686478 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.686449 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6f5cc774-8c4e-43fc-a268-8b99de6e102e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.687041 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.687023 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6f5cc774-8c4e-43fc-a268-8b99de6e102e-config\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.692256 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.692235 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9ft7\" (UniqueName: \"kubernetes.io/projected/6f5cc774-8c4e-43fc-a268-8b99de6e102e-kube-api-access-k9ft7\") pod \"prometheus-k8s-0\" (UID: \"6f5cc774-8c4e-43fc-a268-8b99de6e102e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.804253 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.804217 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:15:03.957364 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:03.957293 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 16:15:03.960438 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:15:03.960409 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5cc774_8c4e_43fc_a268_8b99de6e102e.slice/crio-82ea2f676a76f5254bd75d2ae0601063344f155196e1d451490f0c96a7aeb7b2 WatchSource:0}: Error finding container 82ea2f676a76f5254bd75d2ae0601063344f155196e1d451490f0c96a7aeb7b2: Status 404 returned error can't find the container with id 82ea2f676a76f5254bd75d2ae0601063344f155196e1d451490f0c96a7aeb7b2 Jul 22 16:15:04.410126 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:04.410090 2565 generic.go:356] "Generic (PLEG): container finished" podID="6f5cc774-8c4e-43fc-a268-8b99de6e102e" containerID="a497c342c71d45297bae82c6485d26fd65014f3f270b726a9bbd1e7a2d6acb96" exitCode=0 Jul 22 16:15:04.410126 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:04.410131 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f5cc774-8c4e-43fc-a268-8b99de6e102e","Type":"ContainerDied","Data":"a497c342c71d45297bae82c6485d26fd65014f3f270b726a9bbd1e7a2d6acb96"} Jul 22 16:15:04.410380 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:04.410178 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f5cc774-8c4e-43fc-a268-8b99de6e102e","Type":"ContainerStarted","Data":"82ea2f676a76f5254bd75d2ae0601063344f155196e1d451490f0c96a7aeb7b2"} Jul 22 16:15:05.417271 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:05.417237 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f5cc774-8c4e-43fc-a268-8b99de6e102e","Type":"ContainerStarted","Data":"0535078fc29ccd44df10b6b4d11abfc9de058e39943a0306fea717a643cc90b4"} Jul 22 16:15:05.417685 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:05.417329 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f5cc774-8c4e-43fc-a268-8b99de6e102e","Type":"ContainerStarted","Data":"cfb503f9683429120b0ea28ac470b892f228c5d2846333b132460eb4109e05fc"} Jul 22 16:15:05.417685 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:05.417356 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f5cc774-8c4e-43fc-a268-8b99de6e102e","Type":"ContainerStarted","Data":"0a9bdc5206ab036287cd3a235271b87febd0052a55c943c2813479951e5f969d"} Jul 22 16:15:05.417685 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:05.417380 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f5cc774-8c4e-43fc-a268-8b99de6e102e","Type":"ContainerStarted","Data":"206eaa6fa9e49d3e43aa63d837a6a124081afaa2c1b026b8ea12715ace205022"} Jul 22 16:15:05.417685 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:05.417423 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f5cc774-8c4e-43fc-a268-8b99de6e102e","Type":"ContainerStarted","Data":"ad158ca4634efdfdadf3f6dbe7e63463c8b88cee375afccc0eab93bd37a289de"} Jul 22 16:15:05.417685 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:05.417447 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f5cc774-8c4e-43fc-a268-8b99de6e102e","Type":"ContainerStarted","Data":"0e17ded43d24df05469acf519fec51e1c499d339b8b2b43acb410ffacf26d7a5"} Jul 22 16:15:05.467441 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:05.467365 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.443778648 podStartE2EDuration="2.467347147s" totalImagesPullingTime="0s" totalInitContainerRuntime="23.568499ms" isStatefulPod=false podCreationTimestamp="2026-07-22 16:15:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:15:05.466973299 +0000 UTC m=+202.480140338" watchObservedRunningTime="2026-07-22 16:15:05.467347147 +0000 UTC m=+202.480514185" Jul 22 16:15:08.804708 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:15:08.804666 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:16:03.804644 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:16:03.804595 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:16:03.820338 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:16:03.820316 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:16:04.644367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:16:04.644340 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 16:16:43.392871 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:16:43.392838 2565 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 16:24:57.603188 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:24:57.603141 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-vhsbs"] Jul 22 16:24:57.606431 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:24:57.606407 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-vhsbs" Jul 22 16:24:57.609328 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:24:57.609303 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-ssglr\"" Jul 22 16:24:57.609485 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:24:57.609333 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 22 16:24:57.609485 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:24:57.609425 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 22 16:24:57.614486 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:24:57.614463 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-vhsbs"] Jul 22 16:24:57.726570 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:24:57.726517 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjlpq\" (UniqueName: \"kubernetes.io/projected/71a080cc-c55f-4647-a0db-9b60d8e9f65a-kube-api-access-pjlpq\") pod \"redis-69bbd798b6-vhsbs\" (UID: \"71a080cc-c55f-4647-a0db-9b60d8e9f65a\") " pod="test-ns-feast/redis-69bbd798b6-vhsbs" Jul 22 16:24:57.827982 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:24:57.827953 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjlpq\" (UniqueName: \"kubernetes.io/projected/71a080cc-c55f-4647-a0db-9b60d8e9f65a-kube-api-access-pjlpq\") pod \"redis-69bbd798b6-vhsbs\" (UID: \"71a080cc-c55f-4647-a0db-9b60d8e9f65a\") " pod="test-ns-feast/redis-69bbd798b6-vhsbs" Jul 22 16:24:57.838054 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:24:57.838024 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjlpq\" (UniqueName: \"kubernetes.io/projected/71a080cc-c55f-4647-a0db-9b60d8e9f65a-kube-api-access-pjlpq\") pod \"redis-69bbd798b6-vhsbs\" (UID: \"71a080cc-c55f-4647-a0db-9b60d8e9f65a\") " pod="test-ns-feast/redis-69bbd798b6-vhsbs" Jul 22 16:24:57.917043 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:24:57.917008 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-vhsbs" Jul 22 16:24:58.045240 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:24:58.045207 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-vhsbs"] Jul 22 16:24:58.048107 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:24:58.048082 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71a080cc_c55f_4647_a0db_9b60d8e9f65a.slice/crio-788b6916951fd17e713166f50d7ad17ba86171b4730f44abf31ff76a2b2fe2e0 WatchSource:0}: Error finding container 788b6916951fd17e713166f50d7ad17ba86171b4730f44abf31ff76a2b2fe2e0: Status 404 returned error can't find the container with id 788b6916951fd17e713166f50d7ad17ba86171b4730f44abf31ff76a2b2fe2e0 Jul 22 16:24:58.050023 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:24:58.050003 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 16:24:58.579451 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:24:58.579417 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-vhsbs" event={"ID":"71a080cc-c55f-4647-a0db-9b60d8e9f65a","Type":"ContainerStarted","Data":"788b6916951fd17e713166f50d7ad17ba86171b4730f44abf31ff76a2b2fe2e0"} Jul 22 16:25:00.589423 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:25:00.589365 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-vhsbs" event={"ID":"71a080cc-c55f-4647-a0db-9b60d8e9f65a","Type":"ContainerStarted","Data":"bcfb96f261428378c6c84f871834732edbd32d0a00b31b5837dea9f2355b1cc9"} Jul 22 16:25:00.645122 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:25:00.645067 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-vhsbs" podStartSLOduration=1.890002084 podStartE2EDuration="3.645050202s" totalImagesPullingTime="1.755048118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:24:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:25:00.625982925 +0000 UTC m=+797.639149965" watchObservedRunningTime="2026-07-22 16:25:00.645050202 +0000 UTC m=+797.658217240" Jul 22 16:26:33.445271 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:26:33.445236 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-vhsbs"] Jul 22 16:26:33.445752 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:26:33.445435 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-vhsbs" podUID="71a080cc-c55f-4647-a0db-9b60d8e9f65a" containerName="redis" containerID="cri-o://bcfb96f261428378c6c84f871834732edbd32d0a00b31b5837dea9f2355b1cc9" gracePeriod=30 Jul 22 16:26:33.693936 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:26:33.693912 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-vhsbs" Jul 22 16:26:33.843945 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:26:33.843845 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjlpq\" (UniqueName: \"kubernetes.io/projected/71a080cc-c55f-4647-a0db-9b60d8e9f65a-kube-api-access-pjlpq\") pod \"71a080cc-c55f-4647-a0db-9b60d8e9f65a\" (UID: \"71a080cc-c55f-4647-a0db-9b60d8e9f65a\") " Jul 22 16:26:33.846098 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:26:33.846070 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71a080cc-c55f-4647-a0db-9b60d8e9f65a-kube-api-access-pjlpq" (OuterVolumeSpecName: "kube-api-access-pjlpq") pod "71a080cc-c55f-4647-a0db-9b60d8e9f65a" (UID: "71a080cc-c55f-4647-a0db-9b60d8e9f65a"). InnerVolumeSpecName "kube-api-access-pjlpq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:26:33.945330 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:26:33.945285 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjlpq\" (UniqueName: \"kubernetes.io/projected/71a080cc-c55f-4647-a0db-9b60d8e9f65a-kube-api-access-pjlpq\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:26:33.957568 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:26:33.957539 2565 generic.go:356] "Generic (PLEG): container finished" podID="71a080cc-c55f-4647-a0db-9b60d8e9f65a" containerID="bcfb96f261428378c6c84f871834732edbd32d0a00b31b5837dea9f2355b1cc9" exitCode=0 Jul 22 16:26:33.957713 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:26:33.957584 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-vhsbs" Jul 22 16:26:33.957713 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:26:33.957625 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-vhsbs" event={"ID":"71a080cc-c55f-4647-a0db-9b60d8e9f65a","Type":"ContainerDied","Data":"bcfb96f261428378c6c84f871834732edbd32d0a00b31b5837dea9f2355b1cc9"} Jul 22 16:26:33.957713 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:26:33.957669 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-vhsbs" event={"ID":"71a080cc-c55f-4647-a0db-9b60d8e9f65a","Type":"ContainerDied","Data":"788b6916951fd17e713166f50d7ad17ba86171b4730f44abf31ff76a2b2fe2e0"} Jul 22 16:26:33.957713 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:26:33.957691 2565 scope.go:117] "RemoveContainer" containerID="bcfb96f261428378c6c84f871834732edbd32d0a00b31b5837dea9f2355b1cc9" Jul 22 16:26:33.966085 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:26:33.966064 2565 scope.go:117] "RemoveContainer" containerID="bcfb96f261428378c6c84f871834732edbd32d0a00b31b5837dea9f2355b1cc9" Jul 22 16:26:33.966382 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:26:33.966364 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcfb96f261428378c6c84f871834732edbd32d0a00b31b5837dea9f2355b1cc9\": container with ID starting with bcfb96f261428378c6c84f871834732edbd32d0a00b31b5837dea9f2355b1cc9 not found: ID does not exist" containerID="bcfb96f261428378c6c84f871834732edbd32d0a00b31b5837dea9f2355b1cc9" Jul 22 16:26:33.966483 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:26:33.966422 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcfb96f261428378c6c84f871834732edbd32d0a00b31b5837dea9f2355b1cc9"} err="failed to get container status \"bcfb96f261428378c6c84f871834732edbd32d0a00b31b5837dea9f2355b1cc9\": rpc error: code = NotFound desc = could not find container \"bcfb96f261428378c6c84f871834732edbd32d0a00b31b5837dea9f2355b1cc9\": container with ID starting with bcfb96f261428378c6c84f871834732edbd32d0a00b31b5837dea9f2355b1cc9 not found: ID does not exist" Jul 22 16:26:33.979860 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:26:33.979828 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-vhsbs"] Jul 22 16:26:33.984145 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:26:33.984121 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-vhsbs"] Jul 22 16:26:35.527213 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:26:35.527120 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71a080cc-c55f-4647-a0db-9b60d8e9f65a" path="/var/lib/kubelet/pods/71a080cc-c55f-4647-a0db-9b60d8e9f65a/volumes" Jul 22 16:32:46.631894 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:46.631812 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs"] Jul 22 16:32:46.632374 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:46.632231 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71a080cc-c55f-4647-a0db-9b60d8e9f65a" containerName="redis" Jul 22 16:32:46.632374 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:46.632248 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="71a080cc-c55f-4647-a0db-9b60d8e9f65a" containerName="redis" Jul 22 16:32:46.632374 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:46.632337 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="71a080cc-c55f-4647-a0db-9b60d8e9f65a" containerName="redis" Jul 22 16:32:46.635126 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:46.635105 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs" Jul 22 16:32:46.637935 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:46.637917 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Jul 22 16:32:46.639175 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:46.639157 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-pwgtl\"" Jul 22 16:32:46.639175 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:46.639166 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Jul 22 16:32:46.652443 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:46.652415 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs"] Jul 22 16:32:46.654195 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:46.654174 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjg7p\" (UniqueName: \"kubernetes.io/projected/7d026612-b2b7-4a46-9aff-b899b008d625-kube-api-access-sjg7p\") pod \"feast-operator-controller-manager-7495bcd-xvnhs\" (UID: \"7d026612-b2b7-4a46-9aff-b899b008d625\") " pod="feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs" Jul 22 16:32:46.755429 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:46.755381 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjg7p\" (UniqueName: \"kubernetes.io/projected/7d026612-b2b7-4a46-9aff-b899b008d625-kube-api-access-sjg7p\") pod \"feast-operator-controller-manager-7495bcd-xvnhs\" (UID: \"7d026612-b2b7-4a46-9aff-b899b008d625\") " pod="feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs" Jul 22 16:32:46.787757 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:46.787728 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjg7p\" (UniqueName: \"kubernetes.io/projected/7d026612-b2b7-4a46-9aff-b899b008d625-kube-api-access-sjg7p\") pod \"feast-operator-controller-manager-7495bcd-xvnhs\" (UID: \"7d026612-b2b7-4a46-9aff-b899b008d625\") " pod="feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs" Jul 22 16:32:46.945578 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:46.945532 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs" Jul 22 16:32:47.079658 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:47.079636 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs"] Jul 22 16:32:47.082333 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:32:47.082304 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d026612_b2b7_4a46_9aff_b899b008d625.slice/crio-3c011aa7a82cda234b8a504ae155e1e5e99bccf1468418dd09317947d2835586 WatchSource:0}: Error finding container 3c011aa7a82cda234b8a504ae155e1e5e99bccf1468418dd09317947d2835586: Status 404 returned error can't find the container with id 3c011aa7a82cda234b8a504ae155e1e5e99bccf1468418dd09317947d2835586 Jul 22 16:32:47.084478 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:47.084459 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 16:32:47.383234 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:47.383205 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs" event={"ID":"7d026612-b2b7-4a46-9aff-b899b008d625","Type":"ContainerStarted","Data":"3c011aa7a82cda234b8a504ae155e1e5e99bccf1468418dd09317947d2835586"} Jul 22 16:32:50.395031 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:50.394984 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs" event={"ID":"7d026612-b2b7-4a46-9aff-b899b008d625","Type":"ContainerStarted","Data":"61c95d08ac823212594ca940fa4ef300f00c1a574e4b81c85816318734a29886"} Jul 22 16:32:50.395462 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:32:50.395068 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs" Jul 22 16:33:01.400849 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:01.400820 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs" Jul 22 16:33:01.427375 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:01.427319 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs" podStartSLOduration=13.127105765 podStartE2EDuration="15.427302682s" totalImagesPullingTime="2.300196917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:32:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:32:50.415251171 +0000 UTC m=+1267.428418211" watchObservedRunningTime="2026-07-22 16:33:01.427302682 +0000 UTC m=+1278.440469721" Jul 22 16:33:03.461877 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.461845 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69"] Jul 22 16:33:03.471471 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.471449 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:03.474689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.474460 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jul 22 16:33:03.474689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.474497 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 22 16:33:03.474689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.474563 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jul 22 16:33:03.474689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.474503 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 22 16:33:03.475015 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.474783 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jul 22 16:33:03.475015 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.474816 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-qqqxf\"" Jul 22 16:33:03.475015 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.474868 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jul 22 16:33:03.486255 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.486233 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69"] Jul 22 16:33:03.578987 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.578965 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:03.579114 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.578994 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45d45\" (UniqueName: \"kubernetes.io/projected/2de920eb-d7d0-4da7-892e-1484910ebfe0-kube-api-access-45d45\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:03.579114 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.579016 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:03.579114 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.579036 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:03.579268 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.579130 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:03.579268 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.579243 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2de920eb-d7d0-4da7-892e-1484910ebfe0-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:03.680358 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.680337 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2de920eb-d7d0-4da7-892e-1484910ebfe0-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:03.680490 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.680367 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:03.680490 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.680384 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45d45\" (UniqueName: \"kubernetes.io/projected/2de920eb-d7d0-4da7-892e-1484910ebfe0-kube-api-access-45d45\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:03.680490 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.680427 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:03.680490 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.680456 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:03.680490 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.680480 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:03.680769 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:33:03.680507 2565 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Jul 22 16:33:03.680769 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:33:03.680570 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-ui-tls podName:2de920eb-d7d0-4da7-892e-1484910ebfe0 nodeName:}" failed. No retries permitted until 2026-07-22 16:33:04.180550947 +0000 UTC m=+1281.193717970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-ui-tls") pod "feast-simple-feast-setup-777d6cc998-mvj69" (UID: "2de920eb-d7d0-4da7-892e-1484910ebfe0") : secret "feast-simple-feast-setup-ui-tls" not found Jul 22 16:33:03.680769 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:33:03.680638 2565 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Jul 22 16:33:03.680769 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:33:03.680687 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-registry-tls podName:2de920eb-d7d0-4da7-892e-1484910ebfe0 nodeName:}" failed. No retries permitted until 2026-07-22 16:33:04.18067054 +0000 UTC m=+1281.193837572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-registry-tls") pod "feast-simple-feast-setup-777d6cc998-mvj69" (UID: "2de920eb-d7d0-4da7-892e-1484910ebfe0") : secret "feast-simple-feast-setup-registry-tls" not found Jul 22 16:33:03.681030 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.681013 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2de920eb-d7d0-4da7-892e-1484910ebfe0-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:03.683711 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.683690 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:03.684133 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.684116 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:03.691084 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:03.691056 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45d45\" (UniqueName: \"kubernetes.io/projected/2de920eb-d7d0-4da7-892e-1484910ebfe0-kube-api-access-45d45\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:04.184590 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:04.184557 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:04.184760 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:04.184620 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:04.186964 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:04.186930 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:04.187084 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:04.187044 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-mvj69\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:04.383255 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:04.383227 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:04.520323 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:04.520301 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69"] Jul 22 16:33:04.522717 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:33:04.522692 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2de920eb_d7d0_4da7_892e_1484910ebfe0.slice/crio-110870a75cec35ffb1cb87e7afaaeea5507e8bfd353ab175ddb6cb3b3d6830cf WatchSource:0}: Error finding container 110870a75cec35ffb1cb87e7afaaeea5507e8bfd353ab175ddb6cb3b3d6830cf: Status 404 returned error can't find the container with id 110870a75cec35ffb1cb87e7afaaeea5507e8bfd353ab175ddb6cb3b3d6830cf Jul 22 16:33:05.464550 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:05.464506 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" event={"ID":"2de920eb-d7d0-4da7-892e-1484910ebfe0","Type":"ContainerStarted","Data":"110870a75cec35ffb1cb87e7afaaeea5507e8bfd353ab175ddb6cb3b3d6830cf"} Jul 22 16:33:18.528694 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:18.527921 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" event={"ID":"2de920eb-d7d0-4da7-892e-1484910ebfe0","Type":"ContainerStarted","Data":"ad70e17362b750a18b0e63b4ebba6e4c1a0c9260160ea7972c68e00a46f1131c"} Jul 22 16:33:23.551303 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:23.551269 2565 generic.go:356] "Generic (PLEG): container finished" podID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerID="ad70e17362b750a18b0e63b4ebba6e4c1a0c9260160ea7972c68e00a46f1131c" exitCode=0 Jul 22 16:33:23.551725 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:23.551326 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" event={"ID":"2de920eb-d7d0-4da7-892e-1484910ebfe0","Type":"ContainerDied","Data":"ad70e17362b750a18b0e63b4ebba6e4c1a0c9260160ea7972c68e00a46f1131c"} Jul 22 16:33:24.558009 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:24.557960 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" event={"ID":"2de920eb-d7d0-4da7-892e-1484910ebfe0","Type":"ContainerStarted","Data":"6efdc847d73e4c0382280e42f05e60f4893c5e8ec63b3a030892b63c4d7d4c15"} Jul 22 16:33:29.579602 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:29.579563 2565 generic.go:356] "Generic (PLEG): container finished" podID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerID="6efdc847d73e4c0382280e42f05e60f4893c5e8ec63b3a030892b63c4d7d4c15" exitCode=0 Jul 22 16:33:29.579982 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:29.579625 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" event={"ID":"2de920eb-d7d0-4da7-892e-1484910ebfe0","Type":"ContainerDied","Data":"6efdc847d73e4c0382280e42f05e60f4893c5e8ec63b3a030892b63c4d7d4c15"} Jul 22 16:33:30.589459 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:30.589412 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" event={"ID":"2de920eb-d7d0-4da7-892e-1484910ebfe0","Type":"ContainerStarted","Data":"6339a03816339f8288641f8ba047fbec55bdcb27efdf7f81731e37fafac62c99"} Jul 22 16:33:30.589950 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:30.589481 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" event={"ID":"2de920eb-d7d0-4da7-892e-1484910ebfe0","Type":"ContainerStarted","Data":"de6fdd4dde376fb4c0dd3612658e3ea38fa068a1814ccc41465bf6524cdfd569"} Jul 22 16:33:30.589950 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:30.589502 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" event={"ID":"2de920eb-d7d0-4da7-892e-1484910ebfe0","Type":"ContainerStarted","Data":"1c627a3b99a3748f5e23f46383f3da6077420b50e8f9b245fa6fa3bd31862e5e"} Jul 22 16:33:30.589950 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:30.589519 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" event={"ID":"2de920eb-d7d0-4da7-892e-1484910ebfe0","Type":"ContainerStarted","Data":"7bf6d28cb3333900f291814b28c2f4c9f18258667a273392f39d55affc88ab3c"} Jul 22 16:33:30.620584 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:30.620529 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podStartSLOduration=13.881129095 podStartE2EDuration="27.620508219s" totalImagesPullingTime="13.739379124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:33:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:33:30.617594214 +0000 UTC m=+1307.630761254" watchObservedRunningTime="2026-07-22 16:33:30.620508219 +0000 UTC m=+1307.633675259" Jul 22 16:33:31.384220 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:31.384110 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:31.384220 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:31.384173 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:31.384220 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:31.384192 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:31.384220 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:31.384210 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:31.386536 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:31.386460 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.23:8443: connect: connection refused" Jul 22 16:33:31.386536 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:31.386488 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.23:6571: connect: connection refused" Jul 22 16:33:31.386731 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:31.386489 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.23:8816: connect: connection refused" Jul 22 16:33:31.386731 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:31.386560 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused" Jul 22 16:33:34.383998 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:34.383954 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:34.384546 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:34.384023 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:34.384546 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:34.384255 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.23:8816: connect: connection refused" Jul 22 16:33:34.384546 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:34.384272 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused" Jul 22 16:33:34.384546 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:34.384543 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:34.384775 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:34.384583 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:34.384775 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:34.384638 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.23:8443: connect: connection refused" Jul 22 16:33:34.384946 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:34.384922 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.23:6571: connect: connection refused" Jul 22 16:33:37.384948 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:37.384907 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:37.385427 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:37.385137 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:37.385515 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:37.385426 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:37.385750 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:37.385731 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:37.385812 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:37.385781 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:37.390023 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:37.390008 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:37.632290 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:37.632260 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:33:37.634881 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:33:37.634859 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:34:38.935084 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:38.935046 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69"] Jul 22 16:34:38.935655 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:38.935335 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="registry" containerID="cri-o://7bf6d28cb3333900f291814b28c2f4c9f18258667a273392f39d55affc88ab3c" gracePeriod=30 Jul 22 16:34:38.935655 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:38.935476 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="ui" containerID="cri-o://6339a03816339f8288641f8ba047fbec55bdcb27efdf7f81731e37fafac62c99" gracePeriod=30 Jul 22 16:34:38.935655 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:38.935508 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="online" containerID="cri-o://1c627a3b99a3748f5e23f46383f3da6077420b50e8f9b245fa6fa3bd31862e5e" gracePeriod=30 Jul 22 16:34:38.935655 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:38.935464 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="offline" containerID="cri-o://de6fdd4dde376fb4c0dd3612658e3ea38fa068a1814ccc41465bf6524cdfd569" gracePeriod=30 Jul 22 16:34:39.387551 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.387517 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6"] Jul 22 16:34:39.393018 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.392991 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.397184 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.397153 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-lk7zb\"" Jul 22 16:34:39.409566 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.409540 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6"] Jul 22 16:34:39.504796 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.504759 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c76e534d-7125-429c-a389-f428ed6e3d6e-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.504796 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.504801 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjdqm\" (UniqueName: \"kubernetes.io/projected/c76e534d-7125-429c-a389-f428ed6e3d6e-kube-api-access-qjdqm\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.505061 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.504830 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.505061 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.504852 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.505061 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.504873 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.505061 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.504903 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.606295 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.606268 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.606444 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.606301 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.606444 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.606324 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.606444 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.606365 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.606592 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.606444 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c76e534d-7125-429c-a389-f428ed6e3d6e-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.606592 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.606474 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjdqm\" (UniqueName: \"kubernetes.io/projected/c76e534d-7125-429c-a389-f428ed6e3d6e-kube-api-access-qjdqm\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.606887 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.606854 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c76e534d-7125-429c-a389-f428ed6e3d6e-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.608906 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.608884 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.608979 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.608907 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.608979 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.608916 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.608979 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.608918 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.617044 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.617025 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjdqm\" (UniqueName: \"kubernetes.io/projected/c76e534d-7125-429c-a389-f428ed6e3d6e-kube-api-access-qjdqm\") pod \"feast-simple-feast-setup-777d6cc998-8t7x6\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.710565 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.710533 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:39.850582 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.850558 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6"] Jul 22 16:34:39.852633 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:34:39.852608 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc76e534d_7125_429c_a389_f428ed6e3d6e.slice/crio-a7b4e9d30f3c8b130fad4fcabf073234aebecaa0c64269875f878f82ae4ae1a5 WatchSource:0}: Error finding container a7b4e9d30f3c8b130fad4fcabf073234aebecaa0c64269875f878f82ae4ae1a5: Status 404 returned error can't find the container with id a7b4e9d30f3c8b130fad4fcabf073234aebecaa0c64269875f878f82ae4ae1a5 Jul 22 16:34:39.885966 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.885937 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" event={"ID":"c76e534d-7125-429c-a389-f428ed6e3d6e","Type":"ContainerStarted","Data":"a7b4e9d30f3c8b130fad4fcabf073234aebecaa0c64269875f878f82ae4ae1a5"} Jul 22 16:34:39.889178 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.889156 2565 generic.go:356] "Generic (PLEG): container finished" podID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerID="6339a03816339f8288641f8ba047fbec55bdcb27efdf7f81731e37fafac62c99" exitCode=0 Jul 22 16:34:39.889178 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.889175 2565 generic.go:356] "Generic (PLEG): container finished" podID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerID="1c627a3b99a3748f5e23f46383f3da6077420b50e8f9b245fa6fa3bd31862e5e" exitCode=0 Jul 22 16:34:39.889298 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.889196 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" event={"ID":"2de920eb-d7d0-4da7-892e-1484910ebfe0","Type":"ContainerDied","Data":"6339a03816339f8288641f8ba047fbec55bdcb27efdf7f81731e37fafac62c99"} Jul 22 16:34:39.889298 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:39.889224 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" event={"ID":"2de920eb-d7d0-4da7-892e-1484910ebfe0","Type":"ContainerDied","Data":"1c627a3b99a3748f5e23f46383f3da6077420b50e8f9b245fa6fa3bd31862e5e"} Jul 22 16:34:40.895337 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:40.895291 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" event={"ID":"c76e534d-7125-429c-a389-f428ed6e3d6e","Type":"ContainerStarted","Data":"4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5"} Jul 22 16:34:44.912180 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:44.912150 2565 generic.go:356] "Generic (PLEG): container finished" podID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerID="4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5" exitCode=0 Jul 22 16:34:44.912514 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:44.912202 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" event={"ID":"c76e534d-7125-429c-a389-f428ed6e3d6e","Type":"ContainerDied","Data":"4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5"} Jul 22 16:34:45.917911 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:45.917878 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" event={"ID":"c76e534d-7125-429c-a389-f428ed6e3d6e","Type":"ContainerStarted","Data":"a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547"} Jul 22 16:34:47.632611 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:47.632569 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.23:8443: connect: connection refused" Jul 22 16:34:47.632611 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:47.632596 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused" Jul 22 16:34:50.941124 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:50.941093 2565 generic.go:356] "Generic (PLEG): container finished" podID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerID="a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547" exitCode=0 Jul 22 16:34:50.941561 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:50.941165 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" event={"ID":"c76e534d-7125-429c-a389-f428ed6e3d6e","Type":"ContainerDied","Data":"a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547"} Jul 22 16:34:51.951214 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:51.951156 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" event={"ID":"c76e534d-7125-429c-a389-f428ed6e3d6e","Type":"ContainerStarted","Data":"fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15"} Jul 22 16:34:51.951723 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:51.951232 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" event={"ID":"c76e534d-7125-429c-a389-f428ed6e3d6e","Type":"ContainerStarted","Data":"77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2"} Jul 22 16:34:51.951723 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:51.951253 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" event={"ID":"c76e534d-7125-429c-a389-f428ed6e3d6e","Type":"ContainerStarted","Data":"94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e"} Jul 22 16:34:51.951723 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:51.951271 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" event={"ID":"c76e534d-7125-429c-a389-f428ed6e3d6e","Type":"ContainerStarted","Data":"afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab"} Jul 22 16:34:51.982616 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:51.982549 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" podStartSLOduration=12.982528135999999 podStartE2EDuration="12.982528136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:34:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:34:51.979716477 +0000 UTC m=+1388.992883528" watchObservedRunningTime="2026-07-22 16:34:51.982528136 +0000 UTC m=+1388.995695175" Jul 22 16:34:54.711624 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:54.711578 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:54.712102 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:54.711638 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:54.712102 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:54.711658 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:54.712211 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:54.712106 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:54.713918 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:54.713888 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.24:6571: connect: connection refused" Jul 22 16:34:54.714071 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:54.714030 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.24:8816: connect: connection refused" Jul 22 16:34:54.714152 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:54.714028 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.24:8443: connect: connection refused" Jul 22 16:34:54.714152 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:54.714103 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="online" probeResult="failure" output="Get \"https://10.132.0.24:6567/health\": dial tcp 10.132.0.24:6567: connect: connection refused" Jul 22 16:34:57.632433 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:57.632362 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.23:8443: connect: connection refused" Jul 22 16:34:57.632798 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:57.632465 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused" Jul 22 16:34:57.711974 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:57.711945 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:57.712100 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:57.712043 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:57.712268 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:57.712252 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:57.712522 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:57.712505 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:57.712687 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:57.712666 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:57.712780 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:57.712696 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:57.712981 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:57.712961 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:57.713055 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:57.713011 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:57.713055 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:57.713044 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:57.716688 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:57.716672 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:57.989245 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:57.989173 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:34:57.992251 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:34:57.992228 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:35:07.632697 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:07.632653 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.23:8443: connect: connection refused" Jul 22 16:35:07.633050 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:07.632697 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused" Jul 22 16:35:07.633050 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:07.632788 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:35:07.633050 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:07.632855 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:35:09.031547 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.031523 2565 generic.go:356] "Generic (PLEG): container finished" podID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerID="de6fdd4dde376fb4c0dd3612658e3ea38fa068a1814ccc41465bf6524cdfd569" exitCode=137 Jul 22 16:35:09.031547 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.031545 2565 generic.go:356] "Generic (PLEG): container finished" podID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerID="7bf6d28cb3333900f291814b28c2f4c9f18258667a273392f39d55affc88ab3c" exitCode=137 Jul 22 16:35:09.031846 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.031596 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" event={"ID":"2de920eb-d7d0-4da7-892e-1484910ebfe0","Type":"ContainerDied","Data":"de6fdd4dde376fb4c0dd3612658e3ea38fa068a1814ccc41465bf6524cdfd569"} Jul 22 16:35:09.031846 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.031647 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" event={"ID":"2de920eb-d7d0-4da7-892e-1484910ebfe0","Type":"ContainerDied","Data":"7bf6d28cb3333900f291814b28c2f4c9f18258667a273392f39d55affc88ab3c"} Jul 22 16:35:09.584736 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.584714 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:35:09.739438 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.739346 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-registry-tls\") pod \"2de920eb-d7d0-4da7-892e-1484910ebfe0\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " Jul 22 16:35:09.739438 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.739408 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45d45\" (UniqueName: \"kubernetes.io/projected/2de920eb-d7d0-4da7-892e-1484910ebfe0-kube-api-access-45d45\") pod \"2de920eb-d7d0-4da7-892e-1484910ebfe0\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " Jul 22 16:35:09.739616 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.739466 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2de920eb-d7d0-4da7-892e-1484910ebfe0-feast-data\") pod \"2de920eb-d7d0-4da7-892e-1484910ebfe0\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " Jul 22 16:35:09.739616 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.739509 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-online-tls\") pod \"2de920eb-d7d0-4da7-892e-1484910ebfe0\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " Jul 22 16:35:09.739616 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.739539 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-ui-tls\") pod \"2de920eb-d7d0-4da7-892e-1484910ebfe0\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " Jul 22 16:35:09.739616 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.739570 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-offline-tls\") pod \"2de920eb-d7d0-4da7-892e-1484910ebfe0\" (UID: \"2de920eb-d7d0-4da7-892e-1484910ebfe0\") " Jul 22 16:35:09.740030 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.740000 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2de920eb-d7d0-4da7-892e-1484910ebfe0-feast-data" (OuterVolumeSpecName: "feast-data") pod "2de920eb-d7d0-4da7-892e-1484910ebfe0" (UID: "2de920eb-d7d0-4da7-892e-1484910ebfe0"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:35:09.741721 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.741695 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-online-tls" (OuterVolumeSpecName: "online-tls") pod "2de920eb-d7d0-4da7-892e-1484910ebfe0" (UID: "2de920eb-d7d0-4da7-892e-1484910ebfe0"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:35:09.741806 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.741713 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "2de920eb-d7d0-4da7-892e-1484910ebfe0" (UID: "2de920eb-d7d0-4da7-892e-1484910ebfe0"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:35:09.741843 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.741809 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "2de920eb-d7d0-4da7-892e-1484910ebfe0" (UID: "2de920eb-d7d0-4da7-892e-1484910ebfe0"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:35:09.741974 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.741956 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2de920eb-d7d0-4da7-892e-1484910ebfe0" (UID: "2de920eb-d7d0-4da7-892e-1484910ebfe0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:35:09.742065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.742049 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2de920eb-d7d0-4da7-892e-1484910ebfe0-kube-api-access-45d45" (OuterVolumeSpecName: "kube-api-access-45d45") pod "2de920eb-d7d0-4da7-892e-1484910ebfe0" (UID: "2de920eb-d7d0-4da7-892e-1484910ebfe0"). InnerVolumeSpecName "kube-api-access-45d45". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:35:09.840267 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.840244 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45d45\" (UniqueName: \"kubernetes.io/projected/2de920eb-d7d0-4da7-892e-1484910ebfe0-kube-api-access-45d45\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:35:09.840267 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.840268 2565 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2de920eb-d7d0-4da7-892e-1484910ebfe0-feast-data\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:35:09.840417 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.840283 2565 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-online-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:35:09.840417 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.840293 2565 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-ui-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:35:09.840417 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.840303 2565 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-offline-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:35:09.840417 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:09.840314 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2de920eb-d7d0-4da7-892e-1484910ebfe0-registry-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:35:10.038449 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:10.038379 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" event={"ID":"2de920eb-d7d0-4da7-892e-1484910ebfe0","Type":"ContainerDied","Data":"110870a75cec35ffb1cb87e7afaaeea5507e8bfd353ab175ddb6cb3b3d6830cf"} Jul 22 16:35:10.038449 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:10.038420 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69" Jul 22 16:35:10.038449 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:10.038446 2565 scope.go:117] "RemoveContainer" containerID="6339a03816339f8288641f8ba047fbec55bdcb27efdf7f81731e37fafac62c99" Jul 22 16:35:10.047079 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:10.047056 2565 scope.go:117] "RemoveContainer" containerID="de6fdd4dde376fb4c0dd3612658e3ea38fa068a1814ccc41465bf6524cdfd569" Jul 22 16:35:10.054047 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:10.054030 2565 scope.go:117] "RemoveContainer" containerID="1c627a3b99a3748f5e23f46383f3da6077420b50e8f9b245fa6fa3bd31862e5e" Jul 22 16:35:10.060979 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:10.060964 2565 scope.go:117] "RemoveContainer" containerID="7bf6d28cb3333900f291814b28c2f4c9f18258667a273392f39d55affc88ab3c" Jul 22 16:35:10.065080 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:10.065058 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69"] Jul 22 16:35:10.068418 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:10.068404 2565 scope.go:117] "RemoveContainer" containerID="6efdc847d73e4c0382280e42f05e60f4893c5e8ec63b3a030892b63c4d7d4c15" Jul 22 16:35:10.070860 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:10.070844 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-mvj69"] Jul 22 16:35:10.075269 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:10.075256 2565 scope.go:117] "RemoveContainer" containerID="ad70e17362b750a18b0e63b4ebba6e4c1a0c9260160ea7972c68e00a46f1131c" Jul 22 16:35:11.526629 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:35:11.526601 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" path="/var/lib/kubelet/pods/2de920eb-d7d0-4da7-892e-1484910ebfe0/volumes" Jul 22 16:36:01.198451 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198421 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk"] Jul 22 16:36:01.198983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198766 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="offline" Jul 22 16:36:01.198983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198779 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="offline" Jul 22 16:36:01.198983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198795 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="feast-init" Jul 22 16:36:01.198983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198800 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="feast-init" Jul 22 16:36:01.198983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198807 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="ui" Jul 22 16:36:01.198983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198813 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="ui" Jul 22 16:36:01.198983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198827 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="feast-apply" Jul 22 16:36:01.198983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198835 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="feast-apply" Jul 22 16:36:01.198983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198848 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="registry" Jul 22 16:36:01.198983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198853 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="registry" Jul 22 16:36:01.198983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198861 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="online" Jul 22 16:36:01.198983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198866 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="online" Jul 22 16:36:01.198983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198918 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="online" Jul 22 16:36:01.198983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198927 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="registry" Jul 22 16:36:01.198983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198935 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="offline" Jul 22 16:36:01.198983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.198940 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2de920eb-d7d0-4da7-892e-1484910ebfe0" containerName="ui" Jul 22 16:36:01.203490 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.203472 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.206562 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.206534 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-zc6zl\"" Jul 22 16:36:01.206738 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.206542 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jul 22 16:36:01.206879 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.206534 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jul 22 16:36:01.206994 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.206585 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jul 22 16:36:01.206994 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.206586 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jul 22 16:36:01.206994 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.206597 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jul 22 16:36:01.207156 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.206631 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jul 22 16:36:01.214328 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.214308 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk"] Jul 22 16:36:01.385581 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.385559 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzh4v\" (UniqueName: \"kubernetes.io/projected/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-kube-api-access-qzh4v\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.385686 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.385591 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.385686 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.385623 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.385686 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.385666 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.385802 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.385692 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.385802 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.385722 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.486825 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.486771 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.486825 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.486813 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.486956 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.486841 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.486956 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.486865 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.486956 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.486906 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzh4v\" (UniqueName: \"kubernetes.io/projected/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-kube-api-access-qzh4v\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.487079 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:36:01.486998 2565 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Jul 22 16:36:01.487079 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.487042 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.487079 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:36:01.487069 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-online-tls podName:ad1bb2dc-ae53-4346-9f82-0b387c28b3db nodeName:}" failed. No retries permitted until 2026-07-22 16:36:01.987049089 +0000 UTC m=+1459.000216123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-online-tls") pod "feast-simple-feast-remote-setup-db6955c6c-qzpqk" (UID: "ad1bb2dc-ae53-4346-9f82-0b387c28b3db") : secret "feast-simple-feast-remote-setup-online-tls" not found Jul 22 16:36:01.487244 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.487190 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.487656 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.487632 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.489280 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.489258 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.489372 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.489317 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.499496 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.499473 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzh4v\" (UniqueName: \"kubernetes.io/projected/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-kube-api-access-qzh4v\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.991318 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:01.991283 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:01.991493 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:36:01.991430 2565 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Jul 22 16:36:01.991548 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:36:01.991498 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-online-tls podName:ad1bb2dc-ae53-4346-9f82-0b387c28b3db nodeName:}" failed. No retries permitted until 2026-07-22 16:36:02.991483809 +0000 UTC m=+1460.004650827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-online-tls") pod "feast-simple-feast-remote-setup-db6955c6c-qzpqk" (UID: "ad1bb2dc-ae53-4346-9f82-0b387c28b3db") : secret "feast-simple-feast-remote-setup-online-tls" not found Jul 22 16:36:02.999615 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:02.999584 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:03.001769 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:03.001742 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-qzpqk\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:03.012784 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:03.012764 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:03.144035 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:03.144013 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk"] Jul 22 16:36:03.145375 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:36:03.145353 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad1bb2dc_ae53_4346_9f82_0b387c28b3db.slice/crio-48d711338ace5b4e4a5b90858796c0201de5b3ee68d3c37c61fb8046a86c7ee6 WatchSource:0}: Error finding container 48d711338ace5b4e4a5b90858796c0201de5b3ee68d3c37c61fb8046a86c7ee6: Status 404 returned error can't find the container with id 48d711338ace5b4e4a5b90858796c0201de5b3ee68d3c37c61fb8046a86c7ee6 Jul 22 16:36:03.230711 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:03.230685 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" event={"ID":"ad1bb2dc-ae53-4346-9f82-0b387c28b3db","Type":"ContainerStarted","Data":"71e2292dec383b9e0672e540077a5e6618ab3d3a9612aee959ac6057ea03aa50"} Jul 22 16:36:03.230797 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:03.230722 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" event={"ID":"ad1bb2dc-ae53-4346-9f82-0b387c28b3db","Type":"ContainerStarted","Data":"48d711338ace5b4e4a5b90858796c0201de5b3ee68d3c37c61fb8046a86c7ee6"} Jul 22 16:36:08.251258 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:08.251218 2565 generic.go:356] "Generic (PLEG): container finished" podID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerID="71e2292dec383b9e0672e540077a5e6618ab3d3a9612aee959ac6057ea03aa50" exitCode=0 Jul 22 16:36:08.251552 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:08.251292 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" event={"ID":"ad1bb2dc-ae53-4346-9f82-0b387c28b3db","Type":"ContainerDied","Data":"71e2292dec383b9e0672e540077a5e6618ab3d3a9612aee959ac6057ea03aa50"} Jul 22 16:36:09.257282 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:09.257246 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" event={"ID":"ad1bb2dc-ae53-4346-9f82-0b387c28b3db","Type":"ContainerStarted","Data":"e6c4dba2fa76cc75f9d584ba488fed71b24a49c1cf8176bb09efa500092fbdb2"} Jul 22 16:36:14.277015 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:14.276988 2565 generic.go:356] "Generic (PLEG): container finished" podID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerID="e6c4dba2fa76cc75f9d584ba488fed71b24a49c1cf8176bb09efa500092fbdb2" exitCode=0 Jul 22 16:36:14.277339 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:14.277047 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" event={"ID":"ad1bb2dc-ae53-4346-9f82-0b387c28b3db","Type":"ContainerDied","Data":"e6c4dba2fa76cc75f9d584ba488fed71b24a49c1cf8176bb09efa500092fbdb2"} Jul 22 16:36:15.285167 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:15.285130 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" event={"ID":"ad1bb2dc-ae53-4346-9f82-0b387c28b3db","Type":"ContainerStarted","Data":"90620e7fa9f37de6ac56047ed28e6a16d12c7fed4c88aaeff94c975f7071b185"} Jul 22 16:36:15.285562 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:15.285190 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" event={"ID":"ad1bb2dc-ae53-4346-9f82-0b387c28b3db","Type":"ContainerStarted","Data":"e523a5b2867831c14b987fbeda7cc480143e35ed0cddb8aa9e90d43bf26072c4"} Jul 22 16:36:15.285562 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:15.285209 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" event={"ID":"ad1bb2dc-ae53-4346-9f82-0b387c28b3db","Type":"ContainerStarted","Data":"6af0be5602907e50eb31cbeed2af941c95e5eab8794fb98eef813d8ba53b1b0e"} Jul 22 16:36:15.312953 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:15.312899 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" podStartSLOduration=14.312879456 podStartE2EDuration="14.312879456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:36:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:36:15.312172273 +0000 UTC m=+1472.325339312" watchObservedRunningTime="2026-07-22 16:36:15.312879456 +0000 UTC m=+1472.326046494" Jul 22 16:36:18.013311 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:18.013270 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:18.013788 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:18.013336 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:18.013788 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:18.013356 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:18.014812 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:18.014778 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.25:8816: connect: connection refused" Jul 22 16:36:18.014942 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:18.014815 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.25:8443: connect: connection refused" Jul 22 16:36:18.014942 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:18.014823 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="online" probeResult="failure" output="Get \"https://10.132.0.25:6567/health\": dial tcp 10.132.0.25:6567: connect: connection refused" Jul 22 16:36:21.013572 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:21.013543 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:21.013952 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:21.013719 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:21.014084 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:21.014065 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:21.014140 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:21.014106 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:21.014408 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:21.014360 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:21.014540 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:21.014521 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:21.020029 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:21.020010 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:21.312184 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:21.312123 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:36:21.316242 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:36:21.316224 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:37:22.523802 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:22.523761 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk"] Jul 22 16:37:22.524339 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:22.524018 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="online" containerID="cri-o://6af0be5602907e50eb31cbeed2af941c95e5eab8794fb98eef813d8ba53b1b0e" gracePeriod=30 Jul 22 16:37:22.524339 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:22.524039 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="ui" containerID="cri-o://90620e7fa9f37de6ac56047ed28e6a16d12c7fed4c88aaeff94c975f7071b185" gracePeriod=30 Jul 22 16:37:22.524339 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:22.524035 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="offline" containerID="cri-o://e523a5b2867831c14b987fbeda7cc480143e35ed0cddb8aa9e90d43bf26072c4" gracePeriod=30 Jul 22 16:37:22.701684 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:22.701654 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6"] Jul 22 16:37:22.701939 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:22.701891 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="registry" containerID="cri-o://afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab" gracePeriod=30 Jul 22 16:37:22.702065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:22.701934 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="offline" containerID="cri-o://77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2" gracePeriod=30 Jul 22 16:37:22.702065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:22.701933 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="ui" containerID="cri-o://fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15" gracePeriod=30 Jul 22 16:37:22.702065 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:22.701933 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="online" containerID="cri-o://94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e" gracePeriod=30 Jul 22 16:37:23.539243 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:23.539171 2565 generic.go:356] "Generic (PLEG): container finished" podID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerID="fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15" exitCode=0 Jul 22 16:37:23.539243 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:23.539196 2565 generic.go:356] "Generic (PLEG): container finished" podID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerID="94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e" exitCode=0 Jul 22 16:37:23.539624 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:23.539247 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" event={"ID":"c76e534d-7125-429c-a389-f428ed6e3d6e","Type":"ContainerDied","Data":"fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15"} Jul 22 16:37:23.539624 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:23.539284 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" event={"ID":"c76e534d-7125-429c-a389-f428ed6e3d6e","Type":"ContainerDied","Data":"94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e"} Jul 22 16:37:23.541714 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:23.541693 2565 generic.go:356] "Generic (PLEG): container finished" podID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerID="90620e7fa9f37de6ac56047ed28e6a16d12c7fed4c88aaeff94c975f7071b185" exitCode=0 Jul 22 16:37:23.541714 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:23.541712 2565 generic.go:356] "Generic (PLEG): container finished" podID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerID="6af0be5602907e50eb31cbeed2af941c95e5eab8794fb98eef813d8ba53b1b0e" exitCode=0 Jul 22 16:37:23.541840 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:23.541738 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" event={"ID":"ad1bb2dc-ae53-4346-9f82-0b387c28b3db","Type":"ContainerDied","Data":"90620e7fa9f37de6ac56047ed28e6a16d12c7fed4c88aaeff94c975f7071b185"} Jul 22 16:37:23.541840 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:23.541760 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" event={"ID":"ad1bb2dc-ae53-4346-9f82-0b387c28b3db","Type":"ContainerDied","Data":"6af0be5602907e50eb31cbeed2af941c95e5eab8794fb98eef813d8ba53b1b0e"} Jul 22 16:37:27.713323 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:27.713284 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.24:8443: connect: connection refused" Jul 22 16:37:27.989364 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:27.989296 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="online" probeResult="failure" output="Get \"https://10.132.0.24:6567/health\": dial tcp 10.132.0.24:6567: connect: connection refused" Jul 22 16:37:31.014410 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:31.014349 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.25:8443: connect: connection refused" Jul 22 16:37:31.313022 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:31.312956 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="online" probeResult="failure" output="Get \"https://10.132.0.25:6567/health\": dial tcp 10.132.0.25:6567: connect: connection refused" Jul 22 16:37:37.713188 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:37.713140 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.24:8443: connect: connection refused" Jul 22 16:37:37.989507 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:37.989436 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="online" probeResult="failure" output="Get \"https://10.132.0.24:6567/health\": dial tcp 10.132.0.24:6567: connect: connection refused" Jul 22 16:37:41.014660 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:41.014621 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.25:8443: connect: connection refused" Jul 22 16:37:41.312675 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:41.312605 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="online" probeResult="failure" output="Get \"https://10.132.0.25:6567/health\": dial tcp 10.132.0.25:6567: connect: connection refused" Jul 22 16:37:47.713078 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:47.713030 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.24:8443: connect: connection refused" Jul 22 16:37:47.713503 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:47.713163 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:37:47.990172 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:47.990110 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="online" probeResult="failure" output="Get \"https://10.132.0.24:6567/health\": dial tcp 10.132.0.24:6567: connect: connection refused" Jul 22 16:37:47.990287 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:47.990188 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:37:51.014820 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:51.014778 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.25:8443: connect: connection refused" Jul 22 16:37:51.015225 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:51.014898 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:37:51.313196 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:51.313130 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="online" probeResult="failure" output="Get \"https://10.132.0.25:6567/health\": dial tcp 10.132.0.25:6567: connect: connection refused" Jul 22 16:37:51.313309 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:51.313217 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:37:52.644354 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.644321 2565 generic.go:356] "Generic (PLEG): container finished" podID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerID="e523a5b2867831c14b987fbeda7cc480143e35ed0cddb8aa9e90d43bf26072c4" exitCode=137 Jul 22 16:37:52.644679 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.644412 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" event={"ID":"ad1bb2dc-ae53-4346-9f82-0b387c28b3db","Type":"ContainerDied","Data":"e523a5b2867831c14b987fbeda7cc480143e35ed0cddb8aa9e90d43bf26072c4"} Jul 22 16:37:52.699175 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.699150 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:37:52.865351 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.865325 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-offline-tls\") pod \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " Jul 22 16:37:52.865494 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.865381 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-feast-data\") pod \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " Jul 22 16:37:52.865494 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.865447 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzh4v\" (UniqueName: \"kubernetes.io/projected/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-kube-api-access-qzh4v\") pod \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " Jul 22 16:37:52.865494 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.865482 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-ui-tls\") pod \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " Jul 22 16:37:52.865647 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.865556 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-online-tls\") pod \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " Jul 22 16:37:52.865647 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.865583 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-registry-tls\") pod \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\" (UID: \"ad1bb2dc-ae53-4346-9f82-0b387c28b3db\") " Jul 22 16:37:52.866015 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.865990 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ad1bb2dc-ae53-4346-9f82-0b387c28b3db" (UID: "ad1bb2dc-ae53-4346-9f82-0b387c28b3db"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:37:52.866254 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.866232 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-feast-data" (OuterVolumeSpecName: "feast-data") pod "ad1bb2dc-ae53-4346-9f82-0b387c28b3db" (UID: "ad1bb2dc-ae53-4346-9f82-0b387c28b3db"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:37:52.867556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.867535 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "ad1bb2dc-ae53-4346-9f82-0b387c28b3db" (UID: "ad1bb2dc-ae53-4346-9f82-0b387c28b3db"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:37:52.867632 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.867561 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "ad1bb2dc-ae53-4346-9f82-0b387c28b3db" (UID: "ad1bb2dc-ae53-4346-9f82-0b387c28b3db"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:37:52.868050 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.868026 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-kube-api-access-qzh4v" (OuterVolumeSpecName: "kube-api-access-qzh4v") pod "ad1bb2dc-ae53-4346-9f82-0b387c28b3db" (UID: "ad1bb2dc-ae53-4346-9f82-0b387c28b3db"). InnerVolumeSpecName "kube-api-access-qzh4v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:37:52.868050 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.868026 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-online-tls" (OuterVolumeSpecName: "online-tls") pod "ad1bb2dc-ae53-4346-9f82-0b387c28b3db" (UID: "ad1bb2dc-ae53-4346-9f82-0b387c28b3db"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:37:52.966202 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.966176 2565 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-feast-data\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:37:52.966202 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.966196 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzh4v\" (UniqueName: \"kubernetes.io/projected/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-kube-api-access-qzh4v\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:37:52.966202 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.966206 2565 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-ui-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:37:52.966355 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.966233 2565 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-online-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:37:52.966355 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.966242 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-registry-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:37:52.966355 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:52.966249 2565 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ad1bb2dc-ae53-4346-9f82-0b387c28b3db-offline-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:37:53.340842 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.340820 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:37:53.469693 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.469637 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjdqm\" (UniqueName: \"kubernetes.io/projected/c76e534d-7125-429c-a389-f428ed6e3d6e-kube-api-access-qjdqm\") pod \"c76e534d-7125-429c-a389-f428ed6e3d6e\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " Jul 22 16:37:53.469693 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.469665 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-registry-tls\") pod \"c76e534d-7125-429c-a389-f428ed6e3d6e\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " Jul 22 16:37:53.469880 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.469710 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-offline-tls\") pod \"c76e534d-7125-429c-a389-f428ed6e3d6e\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " Jul 22 16:37:53.469880 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.469725 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-online-tls\") pod \"c76e534d-7125-429c-a389-f428ed6e3d6e\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " Jul 22 16:37:53.469880 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.469758 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c76e534d-7125-429c-a389-f428ed6e3d6e-feast-data\") pod \"c76e534d-7125-429c-a389-f428ed6e3d6e\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " Jul 22 16:37:53.469880 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.469849 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-ui-tls\") pod \"c76e534d-7125-429c-a389-f428ed6e3d6e\" (UID: \"c76e534d-7125-429c-a389-f428ed6e3d6e\") " Jul 22 16:37:53.470511 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.470338 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c76e534d-7125-429c-a389-f428ed6e3d6e-feast-data" (OuterVolumeSpecName: "feast-data") pod "c76e534d-7125-429c-a389-f428ed6e3d6e" (UID: "c76e534d-7125-429c-a389-f428ed6e3d6e"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:37:53.471826 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.471802 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c76e534d-7125-429c-a389-f428ed6e3d6e-kube-api-access-qjdqm" (OuterVolumeSpecName: "kube-api-access-qjdqm") pod "c76e534d-7125-429c-a389-f428ed6e3d6e" (UID: "c76e534d-7125-429c-a389-f428ed6e3d6e"). InnerVolumeSpecName "kube-api-access-qjdqm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:37:53.471908 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.471827 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c76e534d-7125-429c-a389-f428ed6e3d6e" (UID: "c76e534d-7125-429c-a389-f428ed6e3d6e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:37:53.471908 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.471866 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-online-tls" (OuterVolumeSpecName: "online-tls") pod "c76e534d-7125-429c-a389-f428ed6e3d6e" (UID: "c76e534d-7125-429c-a389-f428ed6e3d6e"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:37:53.472372 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.472347 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "c76e534d-7125-429c-a389-f428ed6e3d6e" (UID: "c76e534d-7125-429c-a389-f428ed6e3d6e"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:37:53.472372 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.472367 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "c76e534d-7125-429c-a389-f428ed6e3d6e" (UID: "c76e534d-7125-429c-a389-f428ed6e3d6e"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:37:53.570579 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.570559 2565 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-ui-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:37:53.570579 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.570582 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjdqm\" (UniqueName: \"kubernetes.io/projected/c76e534d-7125-429c-a389-f428ed6e3d6e-kube-api-access-qjdqm\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:37:53.570690 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.570590 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-registry-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:37:53.570690 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.570598 2565 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-offline-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:37:53.570690 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.570605 2565 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c76e534d-7125-429c-a389-f428ed6e3d6e-online-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:37:53.570690 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.570611 2565 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c76e534d-7125-429c-a389-f428ed6e3d6e-feast-data\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:37:53.650250 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.650227 2565 generic.go:356] "Generic (PLEG): container finished" podID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerID="77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2" exitCode=137 Jul 22 16:37:53.650250 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.650246 2565 generic.go:356] "Generic (PLEG): container finished" podID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerID="afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab" exitCode=137 Jul 22 16:37:53.650652 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.650302 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" Jul 22 16:37:53.650652 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.650317 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" event={"ID":"c76e534d-7125-429c-a389-f428ed6e3d6e","Type":"ContainerDied","Data":"77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2"} Jul 22 16:37:53.650652 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.650362 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" event={"ID":"c76e534d-7125-429c-a389-f428ed6e3d6e","Type":"ContainerDied","Data":"afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab"} Jul 22 16:37:53.650652 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.650378 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6" event={"ID":"c76e534d-7125-429c-a389-f428ed6e3d6e","Type":"ContainerDied","Data":"a7b4e9d30f3c8b130fad4fcabf073234aebecaa0c64269875f878f82ae4ae1a5"} Jul 22 16:37:53.650652 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.650419 2565 scope.go:117] "RemoveContainer" containerID="fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15" Jul 22 16:37:53.653507 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.653487 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" event={"ID":"ad1bb2dc-ae53-4346-9f82-0b387c28b3db","Type":"ContainerDied","Data":"48d711338ace5b4e4a5b90858796c0201de5b3ee68d3c37c61fb8046a86c7ee6"} Jul 22 16:37:53.653633 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.653512 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk" Jul 22 16:37:53.659954 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.659937 2565 scope.go:117] "RemoveContainer" containerID="77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2" Jul 22 16:37:53.667122 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.667106 2565 scope.go:117] "RemoveContainer" containerID="94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e" Jul 22 16:37:53.674077 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.674061 2565 scope.go:117] "RemoveContainer" containerID="afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab" Jul 22 16:37:53.676689 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.676667 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6"] Jul 22 16:37:53.681360 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.681338 2565 scope.go:117] "RemoveContainer" containerID="a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547" Jul 22 16:37:53.682035 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.682014 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-8t7x6"] Jul 22 16:37:53.688057 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.688042 2565 scope.go:117] "RemoveContainer" containerID="4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5" Jul 22 16:37:53.694835 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.694820 2565 scope.go:117] "RemoveContainer" containerID="fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15" Jul 22 16:37:53.695114 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:37:53.695090 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15\": container with ID starting with fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15 not found: ID does not exist" containerID="fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15" Jul 22 16:37:53.695191 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.695113 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15"} err="failed to get container status \"fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15\": rpc error: code = NotFound desc = could not find container \"fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15\": container with ID starting with fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15 not found: ID does not exist" Jul 22 16:37:53.695191 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.695129 2565 scope.go:117] "RemoveContainer" containerID="77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2" Jul 22 16:37:53.695191 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.695135 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk"] Jul 22 16:37:53.695377 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:37:53.695359 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2\": container with ID starting with 77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2 not found: ID does not exist" containerID="77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2" Jul 22 16:37:53.695439 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.695383 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2"} err="failed to get container status \"77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2\": rpc error: code = NotFound desc = could not find container \"77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2\": container with ID starting with 77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2 not found: ID does not exist" Jul 22 16:37:53.695439 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.695422 2565 scope.go:117] "RemoveContainer" containerID="94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e" Jul 22 16:37:53.695709 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:37:53.695691 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e\": container with ID starting with 94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e not found: ID does not exist" containerID="94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e" Jul 22 16:37:53.695755 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.695713 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e"} err="failed to get container status \"94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e\": rpc error: code = NotFound desc = could not find container \"94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e\": container with ID starting with 94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e not found: ID does not exist" Jul 22 16:37:53.695755 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.695727 2565 scope.go:117] "RemoveContainer" containerID="afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab" Jul 22 16:37:53.695965 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:37:53.695948 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab\": container with ID starting with afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab not found: ID does not exist" containerID="afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab" Jul 22 16:37:53.696022 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.695981 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab"} err="failed to get container status \"afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab\": rpc error: code = NotFound desc = could not find container \"afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab\": container with ID starting with afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab not found: ID does not exist" Jul 22 16:37:53.696022 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.696017 2565 scope.go:117] "RemoveContainer" containerID="a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547" Jul 22 16:37:53.696263 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:37:53.696244 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547\": container with ID starting with a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547 not found: ID does not exist" containerID="a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547" Jul 22 16:37:53.696307 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.696269 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547"} err="failed to get container status \"a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547\": rpc error: code = NotFound desc = could not find container \"a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547\": container with ID starting with a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547 not found: ID does not exist" Jul 22 16:37:53.696307 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.696281 2565 scope.go:117] "RemoveContainer" containerID="4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5" Jul 22 16:37:53.696524 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:37:53.696510 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5\": container with ID starting with 4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5 not found: ID does not exist" containerID="4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5" Jul 22 16:37:53.696572 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.696528 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5"} err="failed to get container status \"4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5\": rpc error: code = NotFound desc = could not find container \"4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5\": container with ID starting with 4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5 not found: ID does not exist" Jul 22 16:37:53.696572 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.696540 2565 scope.go:117] "RemoveContainer" containerID="fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15" Jul 22 16:37:53.696746 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.696726 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15"} err="failed to get container status \"fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15\": rpc error: code = NotFound desc = could not find container \"fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15\": container with ID starting with fb2d234d7cf2f7dc8630da47b1ef9fcc14586c265b9c824cbd3f6f7bbbd24d15 not found: ID does not exist" Jul 22 16:37:53.696812 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.696747 2565 scope.go:117] "RemoveContainer" containerID="77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2" Jul 22 16:37:53.696951 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.696935 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2"} err="failed to get container status \"77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2\": rpc error: code = NotFound desc = could not find container \"77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2\": container with ID starting with 77d4a41892f690f2bff1d84729bac864b6078781d77b7cddf1b28674ead99cb2 not found: ID does not exist" Jul 22 16:37:53.697016 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.696952 2565 scope.go:117] "RemoveContainer" containerID="94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e" Jul 22 16:37:53.697158 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.697143 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e"} err="failed to get container status \"94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e\": rpc error: code = NotFound desc = could not find container \"94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e\": container with ID starting with 94429688827ad64fbbf5f13f56d00641821cf3d43f99fe77b0b544ce6d2d780e not found: ID does not exist" Jul 22 16:37:53.697218 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.697160 2565 scope.go:117] "RemoveContainer" containerID="afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab" Jul 22 16:37:53.697365 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.697349 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab"} err="failed to get container status \"afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab\": rpc error: code = NotFound desc = could not find container \"afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab\": container with ID starting with afaaa4bea7fd989aa429883e282aab8bbd0304758b8cc6400e8189bba1d487ab not found: ID does not exist" Jul 22 16:37:53.697431 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.697366 2565 scope.go:117] "RemoveContainer" containerID="a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547" Jul 22 16:37:53.697601 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.697575 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547"} err="failed to get container status \"a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547\": rpc error: code = NotFound desc = could not find container \"a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547\": container with ID starting with a2384168781dbd6f44d30cdc4c68cc1045c596e0494e9ed7d1a44ae6d0b54547 not found: ID does not exist" Jul 22 16:37:53.697601 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.697602 2565 scope.go:117] "RemoveContainer" containerID="4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5" Jul 22 16:37:53.697878 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.697851 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5"} err="failed to get container status \"4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5\": rpc error: code = NotFound desc = could not find container \"4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5\": container with ID starting with 4ffc0d74ec688b6117866485b2aefa520913d217c8cf087c4d89251287860aa5 not found: ID does not exist" Jul 22 16:37:53.697878 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.697877 2565 scope.go:117] "RemoveContainer" containerID="90620e7fa9f37de6ac56047ed28e6a16d12c7fed4c88aaeff94c975f7071b185" Jul 22 16:37:53.699892 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.699873 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-qzpqk"] Jul 22 16:37:53.704961 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.704943 2565 scope.go:117] "RemoveContainer" containerID="e523a5b2867831c14b987fbeda7cc480143e35ed0cddb8aa9e90d43bf26072c4" Jul 22 16:37:53.711678 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.711663 2565 scope.go:117] "RemoveContainer" containerID="6af0be5602907e50eb31cbeed2af941c95e5eab8794fb98eef813d8ba53b1b0e" Jul 22 16:37:53.718614 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.718594 2565 scope.go:117] "RemoveContainer" containerID="e6c4dba2fa76cc75f9d584ba488fed71b24a49c1cf8176bb09efa500092fbdb2" Jul 22 16:37:53.724800 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:53.724786 2565 scope.go:117] "RemoveContainer" containerID="71e2292dec383b9e0672e540077a5e6618ab3d3a9612aee959ac6057ea03aa50" Jul 22 16:37:55.527367 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:55.527337 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" path="/var/lib/kubelet/pods/ad1bb2dc-ae53-4346-9f82-0b387c28b3db/volumes" Jul 22 16:37:55.527851 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:37:55.527836 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" path="/var/lib/kubelet/pods/c76e534d-7125-429c-a389-f428ed6e3d6e/volumes" Jul 22 16:38:12.015371 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:12.015342 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs"] Jul 22 16:38:12.015841 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:12.015550 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs" podUID="7d026612-b2b7-4a46-9aff-b899b008d625" containerName="manager" containerID="cri-o://61c95d08ac823212594ca940fa4ef300f00c1a574e4b81c85816318734a29886" gracePeriod=10 Jul 22 16:38:12.271636 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:12.271587 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs" Jul 22 16:38:12.394161 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:12.394137 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjg7p\" (UniqueName: \"kubernetes.io/projected/7d026612-b2b7-4a46-9aff-b899b008d625-kube-api-access-sjg7p\") pod \"7d026612-b2b7-4a46-9aff-b899b008d625\" (UID: \"7d026612-b2b7-4a46-9aff-b899b008d625\") " Jul 22 16:38:12.396357 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:12.396332 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d026612-b2b7-4a46-9aff-b899b008d625-kube-api-access-sjg7p" (OuterVolumeSpecName: "kube-api-access-sjg7p") pod "7d026612-b2b7-4a46-9aff-b899b008d625" (UID: "7d026612-b2b7-4a46-9aff-b899b008d625"). InnerVolumeSpecName "kube-api-access-sjg7p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:38:12.494753 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:12.494729 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjg7p\" (UniqueName: \"kubernetes.io/projected/7d026612-b2b7-4a46-9aff-b899b008d625-kube-api-access-sjg7p\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:38:12.723317 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:12.723293 2565 generic.go:356] "Generic (PLEG): container finished" podID="7d026612-b2b7-4a46-9aff-b899b008d625" containerID="61c95d08ac823212594ca940fa4ef300f00c1a574e4b81c85816318734a29886" exitCode=0 Jul 22 16:38:12.723452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:12.723337 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs" Jul 22 16:38:12.723452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:12.723374 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs" event={"ID":"7d026612-b2b7-4a46-9aff-b899b008d625","Type":"ContainerDied","Data":"61c95d08ac823212594ca940fa4ef300f00c1a574e4b81c85816318734a29886"} Jul 22 16:38:12.723452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:12.723449 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs" event={"ID":"7d026612-b2b7-4a46-9aff-b899b008d625","Type":"ContainerDied","Data":"3c011aa7a82cda234b8a504ae155e1e5e99bccf1468418dd09317947d2835586"} Jul 22 16:38:12.723618 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:12.723478 2565 scope.go:117] "RemoveContainer" containerID="61c95d08ac823212594ca940fa4ef300f00c1a574e4b81c85816318734a29886" Jul 22 16:38:12.731685 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:12.731668 2565 scope.go:117] "RemoveContainer" containerID="61c95d08ac823212594ca940fa4ef300f00c1a574e4b81c85816318734a29886" Jul 22 16:38:12.731935 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:38:12.731919 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61c95d08ac823212594ca940fa4ef300f00c1a574e4b81c85816318734a29886\": container with ID starting with 61c95d08ac823212594ca940fa4ef300f00c1a574e4b81c85816318734a29886 not found: ID does not exist" containerID="61c95d08ac823212594ca940fa4ef300f00c1a574e4b81c85816318734a29886" Jul 22 16:38:12.731972 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:12.731943 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61c95d08ac823212594ca940fa4ef300f00c1a574e4b81c85816318734a29886"} err="failed to get container status \"61c95d08ac823212594ca940fa4ef300f00c1a574e4b81c85816318734a29886\": rpc error: code = NotFound desc = could not find container \"61c95d08ac823212594ca940fa4ef300f00c1a574e4b81c85816318734a29886\": container with ID starting with 61c95d08ac823212594ca940fa4ef300f00c1a574e4b81c85816318734a29886 not found: ID does not exist" Jul 22 16:38:12.746399 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:12.746373 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs"] Jul 22 16:38:12.748537 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:12.748519 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-xvnhs"] Jul 22 16:38:13.527244 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:13.527208 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d026612-b2b7-4a46-9aff-b899b008d625" path="/var/lib/kubelet/pods/7d026612-b2b7-4a46-9aff-b899b008d625/volumes" Jul 22 16:38:16.884046 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.884019 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-nlwml"] Jul 22 16:38:16.890058 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890024 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="feast-init" Jul 22 16:38:16.890439 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890266 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="feast-init" Jul 22 16:38:16.890543 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890447 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="offline" Jul 22 16:38:16.890543 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890459 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="offline" Jul 22 16:38:16.890543 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890469 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="online" Jul 22 16:38:16.890543 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890477 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="online" Jul 22 16:38:16.890543 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890490 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="ui" Jul 22 16:38:16.890543 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890498 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="ui" Jul 22 16:38:16.890543 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890510 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="online" Jul 22 16:38:16.890543 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890518 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="online" Jul 22 16:38:16.890543 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890532 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="feast-apply" Jul 22 16:38:16.890543 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890538 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="feast-apply" Jul 22 16:38:16.890543 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890547 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="offline" Jul 22 16:38:16.890848 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890555 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="offline" Jul 22 16:38:16.890848 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890563 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="ui" Jul 22 16:38:16.890848 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890571 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="ui" Jul 22 16:38:16.890848 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890582 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d026612-b2b7-4a46-9aff-b899b008d625" containerName="manager" Jul 22 16:38:16.890848 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890734 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d026612-b2b7-4a46-9aff-b899b008d625" containerName="manager" Jul 22 16:38:16.890848 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890747 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="registry" Jul 22 16:38:16.890848 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890756 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="registry" Jul 22 16:38:16.890848 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890771 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="feast-init" Jul 22 16:38:16.890848 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890776 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="feast-init" Jul 22 16:38:16.890848 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890782 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="feast-apply" Jul 22 16:38:16.890848 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890787 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="feast-apply" Jul 22 16:38:16.891165 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890856 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="registry" Jul 22 16:38:16.891165 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890863 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="ui" Jul 22 16:38:16.891165 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890871 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d026612-b2b7-4a46-9aff-b899b008d625" containerName="manager" Jul 22 16:38:16.891165 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890877 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="online" Jul 22 16:38:16.891165 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890883 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="online" Jul 22 16:38:16.891165 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890887 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="offline" Jul 22 16:38:16.891165 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890893 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad1bb2dc-ae53-4346-9f82-0b387c28b3db" containerName="offline" Jul 22 16:38:16.891165 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.890898 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c76e534d-7125-429c-a389-f428ed6e3d6e" containerName="ui" Jul 22 16:38:16.893772 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.893757 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-nlwml" Jul 22 16:38:16.896978 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.896957 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Jul 22 16:38:16.897067 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.897006 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-pwgtl\"" Jul 22 16:38:16.898144 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.898126 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Jul 22 16:38:16.902164 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.902143 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-nlwml"] Jul 22 16:38:16.921974 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:16.921942 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzgtm\" (UniqueName: \"kubernetes.io/projected/fac04f78-30b7-4116-bd48-f18ce1a0addd-kube-api-access-lzgtm\") pod \"feast-operator-controller-manager-7495bcd-nlwml\" (UID: \"fac04f78-30b7-4116-bd48-f18ce1a0addd\") " pod="feast-operator-system/feast-operator-controller-manager-7495bcd-nlwml" Jul 22 16:38:17.022997 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:17.022971 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzgtm\" (UniqueName: \"kubernetes.io/projected/fac04f78-30b7-4116-bd48-f18ce1a0addd-kube-api-access-lzgtm\") pod \"feast-operator-controller-manager-7495bcd-nlwml\" (UID: \"fac04f78-30b7-4116-bd48-f18ce1a0addd\") " pod="feast-operator-system/feast-operator-controller-manager-7495bcd-nlwml" Jul 22 16:38:17.037948 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:17.037926 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzgtm\" (UniqueName: \"kubernetes.io/projected/fac04f78-30b7-4116-bd48-f18ce1a0addd-kube-api-access-lzgtm\") pod \"feast-operator-controller-manager-7495bcd-nlwml\" (UID: \"fac04f78-30b7-4116-bd48-f18ce1a0addd\") " pod="feast-operator-system/feast-operator-controller-manager-7495bcd-nlwml" Jul 22 16:38:17.204014 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:17.203961 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-nlwml" Jul 22 16:38:17.330504 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:17.330481 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7495bcd-nlwml"] Jul 22 16:38:17.333045 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:38:17.333021 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfac04f78_30b7_4116_bd48_f18ce1a0addd.slice/crio-c485e568f99a8fe7c317594e2caf47dd0c9a9928f1e0312fd4ac54efcfa332a7 WatchSource:0}: Error finding container c485e568f99a8fe7c317594e2caf47dd0c9a9928f1e0312fd4ac54efcfa332a7: Status 404 returned error can't find the container with id c485e568f99a8fe7c317594e2caf47dd0c9a9928f1e0312fd4ac54efcfa332a7 Jul 22 16:38:17.743294 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:17.743258 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-nlwml" event={"ID":"fac04f78-30b7-4116-bd48-f18ce1a0addd","Type":"ContainerStarted","Data":"2fb7e2a374655d5003b66164cfd188aed01aa859902076909ddaeb7dd5349cbd"} Jul 22 16:38:17.743461 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:17.743308 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-nlwml" event={"ID":"fac04f78-30b7-4116-bd48-f18ce1a0addd","Type":"ContainerStarted","Data":"c485e568f99a8fe7c317594e2caf47dd0c9a9928f1e0312fd4ac54efcfa332a7"} Jul 22 16:38:17.743461 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:17.743376 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-nlwml" Jul 22 16:38:17.762838 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:17.762795 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-nlwml" podStartSLOduration=1.762783834 podStartE2EDuration="1.762783834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:38:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:38:17.761616402 +0000 UTC m=+1594.774783438" watchObservedRunningTime="2026-07-22 16:38:17.762783834 +0000 UTC m=+1594.775950871" Jul 22 16:38:28.749382 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:28.749351 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-7495bcd-nlwml" Jul 22 16:38:34.407970 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.407935 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn"] Jul 22 16:38:34.412400 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.412370 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.415751 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.415732 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jul 22 16:38:34.415976 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.415745 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 22 16:38:34.416136 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.415775 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 22 16:38:34.416230 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.415780 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jul 22 16:38:34.416285 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.415797 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jul 22 16:38:34.416285 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.415831 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jul 22 16:38:34.416410 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.415840 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-rn2p6\"" Jul 22 16:38:34.430946 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.430925 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn"] Jul 22 16:38:34.439869 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.439841 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.439975 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.439886 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.439975 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.439936 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/04b24f59-b06c-4ce8-ad78-72a31a983403-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.439975 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.439969 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.440142 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.440003 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.440142 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.440031 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8xc7\" (UniqueName: \"kubernetes.io/projected/04b24f59-b06c-4ce8-ad78-72a31a983403-kube-api-access-t8xc7\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.541194 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.541169 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/04b24f59-b06c-4ce8-ad78-72a31a983403-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.541294 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.541225 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.541294 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.541258 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.541294 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.541283 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8xc7\" (UniqueName: \"kubernetes.io/projected/04b24f59-b06c-4ce8-ad78-72a31a983403-kube-api-access-t8xc7\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.541484 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.541319 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.541484 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.541347 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.541484 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:38:34.541455 2565 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Jul 22 16:38:34.541635 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:38:34.541520 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-offline-tls podName:04b24f59-b06c-4ce8-ad78-72a31a983403 nodeName:}" failed. No retries permitted until 2026-07-22 16:38:35.041502413 +0000 UTC m=+1612.054669434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-offline-tls") pod "feast-simple-feast-setup-777d6cc998-8slfn" (UID: "04b24f59-b06c-4ce8-ad78-72a31a983403") : secret "feast-simple-feast-setup-offline-tls" not found Jul 22 16:38:34.541635 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.541570 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/04b24f59-b06c-4ce8-ad78-72a31a983403-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.543747 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.543729 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.543891 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.543812 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.543891 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.543871 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:34.553438 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:34.553408 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8xc7\" (UniqueName: \"kubernetes.io/projected/04b24f59-b06c-4ce8-ad78-72a31a983403-kube-api-access-t8xc7\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:35.047312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:35.047235 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:35.049626 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:35.049601 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-8slfn\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:35.322918 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:35.322854 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:35.457898 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:35.457874 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn"] Jul 22 16:38:35.459658 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:38:35.459634 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04b24f59_b06c_4ce8_ad78_72a31a983403.slice/crio-794e8c465b232d5a2fcf850a5083153cf77003739c0923e8eae4250be4aeff45 WatchSource:0}: Error finding container 794e8c465b232d5a2fcf850a5083153cf77003739c0923e8eae4250be4aeff45: Status 404 returned error can't find the container with id 794e8c465b232d5a2fcf850a5083153cf77003739c0923e8eae4250be4aeff45 Jul 22 16:38:35.809573 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:35.809523 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" event={"ID":"04b24f59-b06c-4ce8-ad78-72a31a983403","Type":"ContainerStarted","Data":"567e5bcdb5329438c9ccea820c2b5f16befa399453010f35a039bb283f693657"} Jul 22 16:38:35.809745 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:35.809601 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" event={"ID":"04b24f59-b06c-4ce8-ad78-72a31a983403","Type":"ContainerStarted","Data":"794e8c465b232d5a2fcf850a5083153cf77003739c0923e8eae4250be4aeff45"} Jul 22 16:38:40.830272 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:40.830239 2565 generic.go:356] "Generic (PLEG): container finished" podID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerID="567e5bcdb5329438c9ccea820c2b5f16befa399453010f35a039bb283f693657" exitCode=0 Jul 22 16:38:40.830625 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:40.830307 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" event={"ID":"04b24f59-b06c-4ce8-ad78-72a31a983403","Type":"ContainerDied","Data":"567e5bcdb5329438c9ccea820c2b5f16befa399453010f35a039bb283f693657"} Jul 22 16:38:41.836255 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:41.836222 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" event={"ID":"04b24f59-b06c-4ce8-ad78-72a31a983403","Type":"ContainerStarted","Data":"ca89ad7979b73d46ce227a9dab0652a69f9679d6335481d811f7a78a15696a6f"} Jul 22 16:38:46.858120 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:46.858085 2565 generic.go:356] "Generic (PLEG): container finished" podID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerID="ca89ad7979b73d46ce227a9dab0652a69f9679d6335481d811f7a78a15696a6f" exitCode=0 Jul 22 16:38:46.858457 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:46.858162 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" event={"ID":"04b24f59-b06c-4ce8-ad78-72a31a983403","Type":"ContainerDied","Data":"ca89ad7979b73d46ce227a9dab0652a69f9679d6335481d811f7a78a15696a6f"} Jul 22 16:38:47.868339 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:47.868293 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" event={"ID":"04b24f59-b06c-4ce8-ad78-72a31a983403","Type":"ContainerStarted","Data":"a3535414aa50c0bef0db6fc664dba546d4eb71bfe3420ef610e8ee22c9eb4f36"} Jul 22 16:38:47.868836 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:47.868361 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" event={"ID":"04b24f59-b06c-4ce8-ad78-72a31a983403","Type":"ContainerStarted","Data":"55d4ec7285fbc3e85ec83d97b6789048fb9019abda451309bde9a37ac6379ce0"} Jul 22 16:38:47.868836 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:47.868403 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" event={"ID":"04b24f59-b06c-4ce8-ad78-72a31a983403","Type":"ContainerStarted","Data":"bb024de379f062fdffb99e531382e15ab7af4b5bdbab7bceb1a2ab9628ea26a9"} Jul 22 16:38:47.868836 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:47.868422 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" event={"ID":"04b24f59-b06c-4ce8-ad78-72a31a983403","Type":"ContainerStarted","Data":"b6bda2cfce9807a368cc43699c0e3448063cc8f129542d5e1551969583a731dd"} Jul 22 16:38:47.897733 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:47.897673 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" podStartSLOduration=13.897651793 podStartE2EDuration="13.897651793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:38:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:38:47.896632493 +0000 UTC m=+1624.909799525" watchObservedRunningTime="2026-07-22 16:38:47.897651793 +0000 UTC m=+1624.910818833" Jul 22 16:38:50.324099 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:50.324044 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:50.324692 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:50.324124 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:50.324692 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:50.324147 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:50.324692 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:50.324178 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:50.326121 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:50.326007 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.27:8816: connect: connection refused" Jul 22 16:38:50.326121 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:50.326022 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jul 22 16:38:50.326121 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:50.326005 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.27:6571: connect: connection refused" Jul 22 16:38:50.326435 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:50.326338 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Jul 22 16:38:53.324648 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:53.324620 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:53.325017 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:53.324736 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:53.325017 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:53.324943 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:53.325283 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:53.325260 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:53.325421 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:53.325299 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:53.325421 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:53.325318 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:53.325564 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:53.325508 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:53.325712 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:53.325695 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:53.325788 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:53.325758 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:53.329350 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:53.329333 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:53.898531 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:53.898506 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:38:53.901569 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:38:53.901551 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:39:55.056110 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.056078 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn"] Jul 22 16:39:55.056611 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.056408 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="offline" containerID="cri-o://55d4ec7285fbc3e85ec83d97b6789048fb9019abda451309bde9a37ac6379ce0" gracePeriod=30 Jul 22 16:39:55.056611 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.056425 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="registry" containerID="cri-o://b6bda2cfce9807a368cc43699c0e3448063cc8f129542d5e1551969583a731dd" gracePeriod=30 Jul 22 16:39:55.056611 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.056494 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="online" containerID="cri-o://bb024de379f062fdffb99e531382e15ab7af4b5bdbab7bceb1a2ab9628ea26a9" gracePeriod=30 Jul 22 16:39:55.056611 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.056514 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="ui" containerID="cri-o://a3535414aa50c0bef0db6fc664dba546d4eb71bfe3420ef610e8ee22c9eb4f36" gracePeriod=30 Jul 22 16:39:55.394500 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.394464 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6"] Jul 22 16:39:55.398995 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.398974 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:55.403952 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.403924 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-mwhjx\"" Jul 22 16:39:55.422491 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.422432 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6"] Jul 22 16:39:55.512821 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.512786 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:55.513072 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.513049 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:55.513210 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.513104 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/032e7114-a009-4593-88f4-067394d4ff8b-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:55.513270 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.513220 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:55.513324 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.513266 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:55.513378 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.513349 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qm54\" (UniqueName: \"kubernetes.io/projected/032e7114-a009-4593-88f4-067394d4ff8b-kube-api-access-7qm54\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:55.614872 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.614841 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:55.615010 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.614907 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qm54\" (UniqueName: \"kubernetes.io/projected/032e7114-a009-4593-88f4-067394d4ff8b-kube-api-access-7qm54\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:55.615010 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.614954 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:55.615010 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.614992 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:55.615195 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:39:55.615007 2565 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Jul 22 16:39:55.615195 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.615039 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/032e7114-a009-4593-88f4-067394d4ff8b-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:55.615195 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:39:55.615071 2565 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Jul 22 16:39:55.615195 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:39:55.615087 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-ui-tls podName:032e7114-a009-4593-88f4-067394d4ff8b nodeName:}" failed. No retries permitted until 2026-07-22 16:39:56.115067851 +0000 UTC m=+1693.128234870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-ui-tls") pod "feast-simple-feast-setup-777d6cc998-nqpt6" (UID: "032e7114-a009-4593-88f4-067394d4ff8b") : secret "feast-simple-feast-setup-ui-tls" not found Jul 22 16:39:55.615195 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.615110 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:55.615195 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:39:55.615145 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-online-tls podName:032e7114-a009-4593-88f4-067394d4ff8b nodeName:}" failed. No retries permitted until 2026-07-22 16:39:56.115110234 +0000 UTC m=+1693.128277254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-online-tls") pod "feast-simple-feast-setup-777d6cc998-nqpt6" (UID: "032e7114-a009-4593-88f4-067394d4ff8b") : secret "feast-simple-feast-setup-online-tls" not found Jul 22 16:39:55.615516 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.615446 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/032e7114-a009-4593-88f4-067394d4ff8b-feast-data\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:55.617710 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.617676 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-registry-tls\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:55.617905 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.617888 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-offline-tls\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:55.628001 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:55.627977 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qm54\" (UniqueName: \"kubernetes.io/projected/032e7114-a009-4593-88f4-067394d4ff8b-kube-api-access-7qm54\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:56.118352 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:56.118330 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:56.118754 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:56.118422 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:56.120785 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:56.119761 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" event={"ID":"04b24f59-b06c-4ce8-ad78-72a31a983403","Type":"ContainerDied","Data":"a3535414aa50c0bef0db6fc664dba546d4eb71bfe3420ef610e8ee22c9eb4f36"} Jul 22 16:39:56.120785 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:56.119883 2565 generic.go:356] "Generic (PLEG): container finished" podID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerID="a3535414aa50c0bef0db6fc664dba546d4eb71bfe3420ef610e8ee22c9eb4f36" exitCode=0 Jul 22 16:39:56.120785 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:56.119911 2565 generic.go:356] "Generic (PLEG): container finished" podID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerID="bb024de379f062fdffb99e531382e15ab7af4b5bdbab7bceb1a2ab9628ea26a9" exitCode=0 Jul 22 16:39:56.120785 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:56.119946 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" event={"ID":"04b24f59-b06c-4ce8-ad78-72a31a983403","Type":"ContainerDied","Data":"bb024de379f062fdffb99e531382e15ab7af4b5bdbab7bceb1a2ab9628ea26a9"} Jul 22 16:39:56.121624 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:56.121598 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-ui-tls\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:56.121722 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:56.121606 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-online-tls\") pod \"feast-simple-feast-setup-777d6cc998-nqpt6\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:56.310985 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:56.310958 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:39:56.453822 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:56.453774 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6"] Jul 22 16:39:56.457345 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:39:56.457319 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod032e7114_a009_4593_88f4_067394d4ff8b.slice/crio-9b57c65acc4bb6af86b509d83d5088d4f0bca403062969a16623c88d2f37e631 WatchSource:0}: Error finding container 9b57c65acc4bb6af86b509d83d5088d4f0bca403062969a16623c88d2f37e631: Status 404 returned error can't find the container with id 9b57c65acc4bb6af86b509d83d5088d4f0bca403062969a16623c88d2f37e631 Jul 22 16:39:57.125037 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:57.125004 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" event={"ID":"032e7114-a009-4593-88f4-067394d4ff8b","Type":"ContainerStarted","Data":"286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89"} Jul 22 16:39:57.125037 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:39:57.125043 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" event={"ID":"032e7114-a009-4593-88f4-067394d4ff8b","Type":"ContainerStarted","Data":"9b57c65acc4bb6af86b509d83d5088d4f0bca403062969a16623c88d2f37e631"} Jul 22 16:40:01.140615 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:01.140583 2565 generic.go:356] "Generic (PLEG): container finished" podID="032e7114-a009-4593-88f4-067394d4ff8b" containerID="286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89" exitCode=0 Jul 22 16:40:01.140903 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:01.140649 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" event={"ID":"032e7114-a009-4593-88f4-067394d4ff8b","Type":"ContainerDied","Data":"286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89"} Jul 22 16:40:02.146963 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:02.146929 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" event={"ID":"032e7114-a009-4593-88f4-067394d4ff8b","Type":"ContainerStarted","Data":"bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f"} Jul 22 16:40:03.325582 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:03.325541 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Jul 22 16:40:03.899214 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:03.899178 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jul 22 16:40:07.167867 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:07.167838 2565 generic.go:356] "Generic (PLEG): container finished" podID="032e7114-a009-4593-88f4-067394d4ff8b" containerID="bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f" exitCode=0 Jul 22 16:40:07.168153 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:07.167906 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" event={"ID":"032e7114-a009-4593-88f4-067394d4ff8b","Type":"ContainerDied","Data":"bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f"} Jul 22 16:40:08.177569 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:08.177520 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" event={"ID":"032e7114-a009-4593-88f4-067394d4ff8b","Type":"ContainerStarted","Data":"097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d"} Jul 22 16:40:08.178029 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:08.177590 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" event={"ID":"032e7114-a009-4593-88f4-067394d4ff8b","Type":"ContainerStarted","Data":"205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd"} Jul 22 16:40:08.178029 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:08.177611 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" event={"ID":"032e7114-a009-4593-88f4-067394d4ff8b","Type":"ContainerStarted","Data":"317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c"} Jul 22 16:40:08.178029 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:08.177628 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" event={"ID":"032e7114-a009-4593-88f4-067394d4ff8b","Type":"ContainerStarted","Data":"de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3"} Jul 22 16:40:08.205914 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:08.205853 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podStartSLOduration=13.205831562 podStartE2EDuration="13.205831562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:39:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:40:08.205196974 +0000 UTC m=+1705.218364014" watchObservedRunningTime="2026-07-22 16:40:08.205831562 +0000 UTC m=+1705.218998599" Jul 22 16:40:08.311527 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:08.311379 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:40:08.311527 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:08.311457 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:40:08.311527 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:08.311478 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:40:08.311818 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:08.311616 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:40:08.313788 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:08.313752 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.28:6571: connect: connection refused" Jul 22 16:40:08.313986 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:08.313772 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Jul 22 16:40:08.314108 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:08.313815 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.28:8816: connect: connection refused" Jul 22 16:40:08.314108 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:08.313810 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Jul 22 16:40:11.311784 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:11.311653 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.28:8816: connect: connection refused" Jul 22 16:40:11.311784 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:11.311674 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Jul 22 16:40:11.311784 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:11.311661 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Jul 22 16:40:11.312410 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:11.312237 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.28:6571: connect: connection refused" Jul 22 16:40:13.326234 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:13.326071 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Jul 22 16:40:13.898882 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:13.898848 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jul 22 16:40:14.312640 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:14.312564 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:40:14.312771 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:14.312727 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:40:14.312851 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:14.312840 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:40:14.313231 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:14.313198 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:40:14.313338 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:14.313255 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:40:14.313338 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:14.313274 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:40:14.313496 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:14.313473 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:40:14.313594 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:14.313531 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:40:14.313635 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:14.313617 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:40:14.317630 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:14.317610 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:40:15.211592 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:15.211566 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:40:15.214649 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:15.214628 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:40:23.325912 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:23.325860 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.27:8443: connect: connection refused" Jul 22 16:40:23.326348 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:23.326006 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:40:23.899317 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:23.899280 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jul 22 16:40:23.899503 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:23.899411 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:40:25.259842 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.259813 2565 generic.go:356] "Generic (PLEG): container finished" podID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerID="55d4ec7285fbc3e85ec83d97b6789048fb9019abda451309bde9a37ac6379ce0" exitCode=137 Jul 22 16:40:25.259842 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.259836 2565 generic.go:356] "Generic (PLEG): container finished" podID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerID="b6bda2cfce9807a368cc43699c0e3448063cc8f129542d5e1551969583a731dd" exitCode=137 Jul 22 16:40:25.260228 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.259862 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" event={"ID":"04b24f59-b06c-4ce8-ad78-72a31a983403","Type":"ContainerDied","Data":"55d4ec7285fbc3e85ec83d97b6789048fb9019abda451309bde9a37ac6379ce0"} Jul 22 16:40:25.260228 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.259894 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" event={"ID":"04b24f59-b06c-4ce8-ad78-72a31a983403","Type":"ContainerDied","Data":"b6bda2cfce9807a368cc43699c0e3448063cc8f129542d5e1551969583a731dd"} Jul 22 16:40:25.702556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.702529 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:40:25.769822 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.769787 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-online-tls\") pod \"04b24f59-b06c-4ce8-ad78-72a31a983403\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " Jul 22 16:40:25.770084 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.769840 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/04b24f59-b06c-4ce8-ad78-72a31a983403-feast-data\") pod \"04b24f59-b06c-4ce8-ad78-72a31a983403\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " Jul 22 16:40:25.770084 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.769901 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8xc7\" (UniqueName: \"kubernetes.io/projected/04b24f59-b06c-4ce8-ad78-72a31a983403-kube-api-access-t8xc7\") pod \"04b24f59-b06c-4ce8-ad78-72a31a983403\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " Jul 22 16:40:25.770084 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.769927 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-registry-tls\") pod \"04b24f59-b06c-4ce8-ad78-72a31a983403\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " Jul 22 16:40:25.770084 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.769950 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-ui-tls\") pod \"04b24f59-b06c-4ce8-ad78-72a31a983403\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " Jul 22 16:40:25.770084 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.769991 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-offline-tls\") pod \"04b24f59-b06c-4ce8-ad78-72a31a983403\" (UID: \"04b24f59-b06c-4ce8-ad78-72a31a983403\") " Jul 22 16:40:25.770887 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.770796 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04b24f59-b06c-4ce8-ad78-72a31a983403-feast-data" (OuterVolumeSpecName: "feast-data") pod "04b24f59-b06c-4ce8-ad78-72a31a983403" (UID: "04b24f59-b06c-4ce8-ad78-72a31a983403"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:40:25.772369 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.772344 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04b24f59-b06c-4ce8-ad78-72a31a983403-kube-api-access-t8xc7" (OuterVolumeSpecName: "kube-api-access-t8xc7") pod "04b24f59-b06c-4ce8-ad78-72a31a983403" (UID: "04b24f59-b06c-4ce8-ad78-72a31a983403"). InnerVolumeSpecName "kube-api-access-t8xc7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:40:25.772369 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.772352 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "04b24f59-b06c-4ce8-ad78-72a31a983403" (UID: "04b24f59-b06c-4ce8-ad78-72a31a983403"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:40:25.772532 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.772374 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "04b24f59-b06c-4ce8-ad78-72a31a983403" (UID: "04b24f59-b06c-4ce8-ad78-72a31a983403"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:40:25.772569 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.772522 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "04b24f59-b06c-4ce8-ad78-72a31a983403" (UID: "04b24f59-b06c-4ce8-ad78-72a31a983403"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:40:25.772569 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.772546 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-online-tls" (OuterVolumeSpecName: "online-tls") pod "04b24f59-b06c-4ce8-ad78-72a31a983403" (UID: "04b24f59-b06c-4ce8-ad78-72a31a983403"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:40:25.871001 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.870973 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8xc7\" (UniqueName: \"kubernetes.io/projected/04b24f59-b06c-4ce8-ad78-72a31a983403-kube-api-access-t8xc7\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:40:25.871001 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.871001 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-registry-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:40:25.871173 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.871016 2565 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-ui-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:40:25.871173 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.871028 2565 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-offline-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:40:25.871173 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.871039 2565 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/04b24f59-b06c-4ce8-ad78-72a31a983403-online-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:40:25.871173 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:25.871049 2565 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/04b24f59-b06c-4ce8-ad78-72a31a983403-feast-data\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:40:26.266423 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:26.266316 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" event={"ID":"04b24f59-b06c-4ce8-ad78-72a31a983403","Type":"ContainerDied","Data":"794e8c465b232d5a2fcf850a5083153cf77003739c0923e8eae4250be4aeff45"} Jul 22 16:40:26.266423 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:26.266376 2565 scope.go:117] "RemoveContainer" containerID="a3535414aa50c0bef0db6fc664dba546d4eb71bfe3420ef610e8ee22c9eb4f36" Jul 22 16:40:26.266423 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:26.266329 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn" Jul 22 16:40:26.275965 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:26.275948 2565 scope.go:117] "RemoveContainer" containerID="55d4ec7285fbc3e85ec83d97b6789048fb9019abda451309bde9a37ac6379ce0" Jul 22 16:40:26.283100 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:26.283086 2565 scope.go:117] "RemoveContainer" containerID="bb024de379f062fdffb99e531382e15ab7af4b5bdbab7bceb1a2ab9628ea26a9" Jul 22 16:40:26.290266 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:26.290247 2565 scope.go:117] "RemoveContainer" containerID="b6bda2cfce9807a368cc43699c0e3448063cc8f129542d5e1551969583a731dd" Jul 22 16:40:26.291582 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:26.291562 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn"] Jul 22 16:40:26.295353 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:26.295331 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-8slfn"] Jul 22 16:40:26.297879 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:26.297867 2565 scope.go:117] "RemoveContainer" containerID="ca89ad7979b73d46ce227a9dab0652a69f9679d6335481d811f7a78a15696a6f" Jul 22 16:40:26.304944 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:26.304928 2565 scope.go:117] "RemoveContainer" containerID="567e5bcdb5329438c9ccea820c2b5f16befa399453010f35a039bb283f693657" Jul 22 16:40:27.527460 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:40:27.527425 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" path="/var/lib/kubelet/pods/04b24f59-b06c-4ce8-ad78-72a31a983403/volumes" Jul 22 16:41:16.916220 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916186 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv"] Jul 22 16:41:16.916691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916500 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="registry" Jul 22 16:41:16.916691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916511 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="registry" Jul 22 16:41:16.916691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916523 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="online" Jul 22 16:41:16.916691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916528 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="online" Jul 22 16:41:16.916691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916536 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="feast-init" Jul 22 16:41:16.916691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916541 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="feast-init" Jul 22 16:41:16.916691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916547 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="feast-apply" Jul 22 16:41:16.916691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916552 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="feast-apply" Jul 22 16:41:16.916691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916562 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="offline" Jul 22 16:41:16.916691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916567 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="offline" Jul 22 16:41:16.916691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916574 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="ui" Jul 22 16:41:16.916691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916579 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="ui" Jul 22 16:41:16.916691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916632 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="online" Jul 22 16:41:16.916691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916639 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="ui" Jul 22 16:41:16.916691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916644 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="registry" Jul 22 16:41:16.916691 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.916650 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="04b24f59-b06c-4ce8-ad78-72a31a983403" containerName="offline" Jul 22 16:41:16.919802 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.919784 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:16.922766 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.922739 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jul 22 16:41:16.922888 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.922868 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jul 22 16:41:16.922888 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.922880 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jul 22 16:41:16.923012 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.922961 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jul 22 16:41:16.923012 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.922884 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jul 22 16:41:16.923012 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.922978 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-qklxd\"" Jul 22 16:41:16.923270 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.923256 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jul 22 16:41:16.932735 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.932713 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv"] Jul 22 16:41:16.939105 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.939084 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:16.939191 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.939133 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5f078cf2-64b6-48dc-9884-bf41a9e035af-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:16.939191 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.939161 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:16.939280 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.939247 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7f28\" (UniqueName: \"kubernetes.io/projected/5f078cf2-64b6-48dc-9884-bf41a9e035af-kube-api-access-d7f28\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:16.939335 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.939290 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:16.939403 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:16.939353 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/5f078cf2-64b6-48dc-9884-bf41a9e035af-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:17.040457 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:17.040424 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:17.040558 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:17.040471 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5f078cf2-64b6-48dc-9884-bf41a9e035af-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:17.040558 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:17.040500 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:17.040558 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:17.040528 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7f28\" (UniqueName: \"kubernetes.io/projected/5f078cf2-64b6-48dc-9884-bf41a9e035af-kube-api-access-d7f28\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:17.040720 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:41:17.040568 2565 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Jul 22 16:41:17.040720 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:17.040590 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:17.040720 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:41:17.040625 2565 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Jul 22 16:41:17.040720 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:41:17.040636 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-online-tls podName:5f078cf2-64b6-48dc-9884-bf41a9e035af nodeName:}" failed. No retries permitted until 2026-07-22 16:41:17.54061851 +0000 UTC m=+1774.553785532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-online-tls") pod "feast-simple-feast-remote-setup-db6955c6c-bhqrv" (UID: "5f078cf2-64b6-48dc-9884-bf41a9e035af") : secret "feast-simple-feast-remote-setup-online-tls" not found Jul 22 16:41:17.040720 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:17.040656 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/5f078cf2-64b6-48dc-9884-bf41a9e035af-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:17.040720 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:41:17.040686 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-ui-tls podName:5f078cf2-64b6-48dc-9884-bf41a9e035af nodeName:}" failed. No retries permitted until 2026-07-22 16:41:17.540676114 +0000 UTC m=+1774.553843135 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-ui-tls") pod "feast-simple-feast-remote-setup-db6955c6c-bhqrv" (UID: "5f078cf2-64b6-48dc-9884-bf41a9e035af") : secret "feast-simple-feast-remote-setup-ui-tls" not found Jul 22 16:41:17.040983 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:17.040967 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5f078cf2-64b6-48dc-9884-bf41a9e035af-feast-data\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:17.041321 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:17.041303 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/5f078cf2-64b6-48dc-9884-bf41a9e035af-registry-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:17.043187 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:17.043136 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-offline-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:17.053830 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:17.053813 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7f28\" (UniqueName: \"kubernetes.io/projected/5f078cf2-64b6-48dc-9884-bf41a9e035af-kube-api-access-d7f28\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:17.544084 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:17.544055 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:17.544236 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:17.544096 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:17.546558 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:17.546529 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-online-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:17.546704 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:17.546570 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-ui-tls\") pod \"feast-simple-feast-remote-setup-db6955c6c-bhqrv\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:17.831308 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:17.831232 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:17.967956 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:17.967932 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv"] Jul 22 16:41:17.970086 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:41:17.970052 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f078cf2_64b6_48dc_9884_bf41a9e035af.slice/crio-c3cf2fe37dc3a5288827dc003cf8387df47492556b6a8fbf8e404b389edcc974 WatchSource:0}: Error finding container c3cf2fe37dc3a5288827dc003cf8387df47492556b6a8fbf8e404b389edcc974: Status 404 returned error can't find the container with id c3cf2fe37dc3a5288827dc003cf8387df47492556b6a8fbf8e404b389edcc974 Jul 22 16:41:18.464634 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:18.464592 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" event={"ID":"5f078cf2-64b6-48dc-9884-bf41a9e035af","Type":"ContainerStarted","Data":"69012252362d857ae4aecc9da4c4aad8cec6e6f142c1597efd329d3ee1e0c339"} Jul 22 16:41:18.464828 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:18.464640 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" event={"ID":"5f078cf2-64b6-48dc-9884-bf41a9e035af","Type":"ContainerStarted","Data":"c3cf2fe37dc3a5288827dc003cf8387df47492556b6a8fbf8e404b389edcc974"} Jul 22 16:41:22.482045 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:22.482017 2565 generic.go:356] "Generic (PLEG): container finished" podID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerID="69012252362d857ae4aecc9da4c4aad8cec6e6f142c1597efd329d3ee1e0c339" exitCode=0 Jul 22 16:41:22.482323 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:22.482093 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" event={"ID":"5f078cf2-64b6-48dc-9884-bf41a9e035af","Type":"ContainerDied","Data":"69012252362d857ae4aecc9da4c4aad8cec6e6f142c1597efd329d3ee1e0c339"} Jul 22 16:41:23.487929 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:23.487895 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" event={"ID":"5f078cf2-64b6-48dc-9884-bf41a9e035af","Type":"ContainerStarted","Data":"544bc3f932d9d7bee97ce7b3894ecda45a82ede98864eb40f511d3da0eb22bb3"} Jul 22 16:41:28.506901 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:28.506870 2565 generic.go:356] "Generic (PLEG): container finished" podID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerID="544bc3f932d9d7bee97ce7b3894ecda45a82ede98864eb40f511d3da0eb22bb3" exitCode=0 Jul 22 16:41:28.507241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:28.506943 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" event={"ID":"5f078cf2-64b6-48dc-9884-bf41a9e035af","Type":"ContainerDied","Data":"544bc3f932d9d7bee97ce7b3894ecda45a82ede98864eb40f511d3da0eb22bb3"} Jul 22 16:41:29.515586 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:29.515546 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" event={"ID":"5f078cf2-64b6-48dc-9884-bf41a9e035af","Type":"ContainerStarted","Data":"622512cacd4f9dc6a21407ae4dd7b1fa6a99dd43f4d132918382285afe214606"} Jul 22 16:41:29.515968 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:29.515607 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" event={"ID":"5f078cf2-64b6-48dc-9884-bf41a9e035af","Type":"ContainerStarted","Data":"d422d0af5d4d4ba53993ddaf3625de0444a0a5e65e3d7bae3f5d5b3095399d95"} Jul 22 16:41:29.515968 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:29.515628 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" event={"ID":"5f078cf2-64b6-48dc-9884-bf41a9e035af","Type":"ContainerStarted","Data":"fe42c6a79c3b0d980717bd2bc35b6723c74a9d3b5c41a8cc0c16db2b16ee19aa"} Jul 22 16:41:29.545787 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:29.545734 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" podStartSLOduration=13.545717679 podStartE2EDuration="13.545717679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 16:41:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:41:29.543106641 +0000 UTC m=+1786.556273682" watchObservedRunningTime="2026-07-22 16:41:29.545717679 +0000 UTC m=+1786.558884716" Jul 22 16:41:29.831735 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:29.831630 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:29.831735 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:29.831685 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:29.831735 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:29.831703 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:29.833088 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:29.833051 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Jul 22 16:41:29.833088 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:29.833065 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.29:8816: connect: connection refused" Jul 22 16:41:29.833280 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:29.833084 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Jul 22 16:41:32.831634 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:32.831531 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.29:8816: connect: connection refused" Jul 22 16:41:32.832044 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:32.831532 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Jul 22 16:41:32.832190 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:32.832148 2565 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Jul 22 16:41:35.832743 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:35.832658 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:35.833079 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:35.832762 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:35.833157 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:35.833136 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:35.833217 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:35.833179 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:35.833524 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:35.833502 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:35.833629 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:35.833550 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:35.838967 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:35.838947 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:36.555985 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:36.555958 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:41:36.560652 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:41:36.560623 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:42:37.899885 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:37.899855 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv"] Jul 22 16:42:37.900435 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:37.900048 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="online" containerID="cri-o://fe42c6a79c3b0d980717bd2bc35b6723c74a9d3b5c41a8cc0c16db2b16ee19aa" gracePeriod=30 Jul 22 16:42:37.900435 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:37.900133 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="offline" containerID="cri-o://d422d0af5d4d4ba53993ddaf3625de0444a0a5e65e3d7bae3f5d5b3095399d95" gracePeriod=30 Jul 22 16:42:37.900435 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:37.900159 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="ui" containerID="cri-o://622512cacd4f9dc6a21407ae4dd7b1fa6a99dd43f4d132918382285afe214606" gracePeriod=30 Jul 22 16:42:38.014880 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:38.014849 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6"] Jul 22 16:42:38.015135 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:38.015085 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="registry" containerID="cri-o://de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3" gracePeriod=30 Jul 22 16:42:38.015234 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:38.015141 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="offline" containerID="cri-o://205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd" gracePeriod=30 Jul 22 16:42:38.015234 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:38.015159 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="online" containerID="cri-o://317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c" gracePeriod=30 Jul 22 16:42:38.015458 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:38.015380 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="ui" containerID="cri-o://097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d" gracePeriod=30 Jul 22 16:42:38.790808 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:38.790780 2565 generic.go:356] "Generic (PLEG): container finished" podID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerID="622512cacd4f9dc6a21407ae4dd7b1fa6a99dd43f4d132918382285afe214606" exitCode=0 Jul 22 16:42:38.790808 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:38.790805 2565 generic.go:356] "Generic (PLEG): container finished" podID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerID="fe42c6a79c3b0d980717bd2bc35b6723c74a9d3b5c41a8cc0c16db2b16ee19aa" exitCode=0 Jul 22 16:42:38.790974 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:38.790847 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" event={"ID":"5f078cf2-64b6-48dc-9884-bf41a9e035af","Type":"ContainerDied","Data":"622512cacd4f9dc6a21407ae4dd7b1fa6a99dd43f4d132918382285afe214606"} Jul 22 16:42:38.790974 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:38.790891 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" event={"ID":"5f078cf2-64b6-48dc-9884-bf41a9e035af","Type":"ContainerDied","Data":"fe42c6a79c3b0d980717bd2bc35b6723c74a9d3b5c41a8cc0c16db2b16ee19aa"} Jul 22 16:42:38.793611 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:38.793592 2565 generic.go:356] "Generic (PLEG): container finished" podID="032e7114-a009-4593-88f4-067394d4ff8b" containerID="097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d" exitCode=0 Jul 22 16:42:38.793611 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:38.793608 2565 generic.go:356] "Generic (PLEG): container finished" podID="032e7114-a009-4593-88f4-067394d4ff8b" containerID="317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c" exitCode=0 Jul 22 16:42:38.793730 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:38.793660 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" event={"ID":"032e7114-a009-4593-88f4-067394d4ff8b","Type":"ContainerDied","Data":"097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d"} Jul 22 16:42:38.793730 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:38.793692 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" event={"ID":"032e7114-a009-4593-88f4-067394d4ff8b","Type":"ContainerDied","Data":"317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c"} Jul 22 16:42:44.313290 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:44.313253 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Jul 22 16:42:45.212411 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:45.212357 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Jul 22 16:42:45.833433 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:45.833377 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Jul 22 16:42:46.556621 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:46.556592 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Jul 22 16:42:54.313645 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:54.313598 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Jul 22 16:42:55.212444 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:55.212380 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Jul 22 16:42:55.833485 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:55.833456 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Jul 22 16:42:56.556329 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:42:56.556296 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Jul 22 16:43:04.313828 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:04.313784 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.28:8443: connect: connection refused" Jul 22 16:43:04.314365 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:04.313953 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:43:05.212179 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:05.212097 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="online" probeResult="failure" output="Get \"https://10.132.0.28:6567/health\": dial tcp 10.132.0.28:6567: connect: connection refused" Jul 22 16:43:05.212330 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:05.212205 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:43:05.833373 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:05.833342 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Jul 22 16:43:05.833795 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:05.833467 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:43:06.556631 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:06.556592 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Jul 22 16:43:06.556797 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:06.556702 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:43:08.105556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.105536 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:43:08.131324 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.131304 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-online-tls\") pod \"5f078cf2-64b6-48dc-9884-bf41a9e035af\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " Jul 22 16:43:08.131465 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.131347 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-ui-tls\") pod \"5f078cf2-64b6-48dc-9884-bf41a9e035af\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " Jul 22 16:43:08.131465 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.131416 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/5f078cf2-64b6-48dc-9884-bf41a9e035af-registry-tls\") pod \"5f078cf2-64b6-48dc-9884-bf41a9e035af\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " Jul 22 16:43:08.131576 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.131467 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7f28\" (UniqueName: \"kubernetes.io/projected/5f078cf2-64b6-48dc-9884-bf41a9e035af-kube-api-access-d7f28\") pod \"5f078cf2-64b6-48dc-9884-bf41a9e035af\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " Jul 22 16:43:08.131576 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.131504 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5f078cf2-64b6-48dc-9884-bf41a9e035af-feast-data\") pod \"5f078cf2-64b6-48dc-9884-bf41a9e035af\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " Jul 22 16:43:08.131576 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.131548 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-offline-tls\") pod \"5f078cf2-64b6-48dc-9884-bf41a9e035af\" (UID: \"5f078cf2-64b6-48dc-9884-bf41a9e035af\") " Jul 22 16:43:08.131878 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.131851 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f078cf2-64b6-48dc-9884-bf41a9e035af-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5f078cf2-64b6-48dc-9884-bf41a9e035af" (UID: "5f078cf2-64b6-48dc-9884-bf41a9e035af"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 16:43:08.132073 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.132019 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f078cf2-64b6-48dc-9884-bf41a9e035af-feast-data" (OuterVolumeSpecName: "feast-data") pod "5f078cf2-64b6-48dc-9884-bf41a9e035af" (UID: "5f078cf2-64b6-48dc-9884-bf41a9e035af"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:43:08.134087 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.134060 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-online-tls" (OuterVolumeSpecName: "online-tls") pod "5f078cf2-64b6-48dc-9884-bf41a9e035af" (UID: "5f078cf2-64b6-48dc-9884-bf41a9e035af"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:43:08.134198 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.134110 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "5f078cf2-64b6-48dc-9884-bf41a9e035af" (UID: "5f078cf2-64b6-48dc-9884-bf41a9e035af"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:43:08.134198 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.134124 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f078cf2-64b6-48dc-9884-bf41a9e035af-kube-api-access-d7f28" (OuterVolumeSpecName: "kube-api-access-d7f28") pod "5f078cf2-64b6-48dc-9884-bf41a9e035af" (UID: "5f078cf2-64b6-48dc-9884-bf41a9e035af"). InnerVolumeSpecName "kube-api-access-d7f28". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:43:08.134298 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.134210 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "5f078cf2-64b6-48dc-9884-bf41a9e035af" (UID: "5f078cf2-64b6-48dc-9884-bf41a9e035af"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:43:08.232781 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.232727 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7f28\" (UniqueName: \"kubernetes.io/projected/5f078cf2-64b6-48dc-9884-bf41a9e035af-kube-api-access-d7f28\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:43:08.232781 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.232749 2565 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/5f078cf2-64b6-48dc-9884-bf41a9e035af-feast-data\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:43:08.232781 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.232762 2565 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-offline-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:43:08.232781 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.232774 2565 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-online-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:43:08.232950 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.232785 2565 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/5f078cf2-64b6-48dc-9884-bf41a9e035af-ui-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:43:08.232950 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.232797 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/5f078cf2-64b6-48dc-9884-bf41a9e035af-registry-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:43:08.656312 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.656288 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:43:08.736057 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.736034 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-offline-tls\") pod \"032e7114-a009-4593-88f4-067394d4ff8b\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " Jul 22 16:43:08.736168 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.736081 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qm54\" (UniqueName: \"kubernetes.io/projected/032e7114-a009-4593-88f4-067394d4ff8b-kube-api-access-7qm54\") pod \"032e7114-a009-4593-88f4-067394d4ff8b\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " Jul 22 16:43:08.736168 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.736118 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-ui-tls\") pod \"032e7114-a009-4593-88f4-067394d4ff8b\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " Jul 22 16:43:08.736168 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.736151 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/032e7114-a009-4593-88f4-067394d4ff8b-feast-data\") pod \"032e7114-a009-4593-88f4-067394d4ff8b\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " Jul 22 16:43:08.736319 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.736173 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-online-tls\") pod \"032e7114-a009-4593-88f4-067394d4ff8b\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " Jul 22 16:43:08.736319 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.736203 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-registry-tls\") pod \"032e7114-a009-4593-88f4-067394d4ff8b\" (UID: \"032e7114-a009-4593-88f4-067394d4ff8b\") " Jul 22 16:43:08.736744 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.736706 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/032e7114-a009-4593-88f4-067394d4ff8b-feast-data" (OuterVolumeSpecName: "feast-data") pod "032e7114-a009-4593-88f4-067394d4ff8b" (UID: "032e7114-a009-4593-88f4-067394d4ff8b"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 16:43:08.738207 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.738180 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "032e7114-a009-4593-88f4-067394d4ff8b" (UID: "032e7114-a009-4593-88f4-067394d4ff8b"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:43:08.738313 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.738219 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-online-tls" (OuterVolumeSpecName: "online-tls") pod "032e7114-a009-4593-88f4-067394d4ff8b" (UID: "032e7114-a009-4593-88f4-067394d4ff8b"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:43:08.738313 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.738244 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "032e7114-a009-4593-88f4-067394d4ff8b" (UID: "032e7114-a009-4593-88f4-067394d4ff8b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:43:08.738602 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.738587 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "032e7114-a009-4593-88f4-067394d4ff8b" (UID: "032e7114-a009-4593-88f4-067394d4ff8b"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 16:43:08.738662 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.738612 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/032e7114-a009-4593-88f4-067394d4ff8b-kube-api-access-7qm54" (OuterVolumeSpecName: "kube-api-access-7qm54") pod "032e7114-a009-4593-88f4-067394d4ff8b" (UID: "032e7114-a009-4593-88f4-067394d4ff8b"). InnerVolumeSpecName "kube-api-access-7qm54". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 16:43:08.837561 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.837512 2565 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-online-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:43:08.837561 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.837531 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-registry-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:43:08.837561 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.837541 2565 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-offline-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:43:08.837561 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.837549 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qm54\" (UniqueName: \"kubernetes.io/projected/032e7114-a009-4593-88f4-067394d4ff8b-kube-api-access-7qm54\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:43:08.837727 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.837560 2565 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/032e7114-a009-4593-88f4-067394d4ff8b-ui-tls\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:43:08.837727 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.837575 2565 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/032e7114-a009-4593-88f4-067394d4ff8b-feast-data\") on node \"ip-10-0-129-160.ec2.internal\" DevicePath \"\"" Jul 22 16:43:08.911797 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.911771 2565 generic.go:356] "Generic (PLEG): container finished" podID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerID="d422d0af5d4d4ba53993ddaf3625de0444a0a5e65e3d7bae3f5d5b3095399d95" exitCode=137 Jul 22 16:43:08.911891 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.911846 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" Jul 22 16:43:08.911891 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.911856 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" event={"ID":"5f078cf2-64b6-48dc-9884-bf41a9e035af","Type":"ContainerDied","Data":"d422d0af5d4d4ba53993ddaf3625de0444a0a5e65e3d7bae3f5d5b3095399d95"} Jul 22 16:43:08.911969 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.911902 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv" event={"ID":"5f078cf2-64b6-48dc-9884-bf41a9e035af","Type":"ContainerDied","Data":"c3cf2fe37dc3a5288827dc003cf8387df47492556b6a8fbf8e404b389edcc974"} Jul 22 16:43:08.911969 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.911930 2565 scope.go:117] "RemoveContainer" containerID="622512cacd4f9dc6a21407ae4dd7b1fa6a99dd43f4d132918382285afe214606" Jul 22 16:43:08.914833 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.914812 2565 generic.go:356] "Generic (PLEG): container finished" podID="032e7114-a009-4593-88f4-067394d4ff8b" containerID="205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd" exitCode=137 Jul 22 16:43:08.914833 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.914833 2565 generic.go:356] "Generic (PLEG): container finished" podID="032e7114-a009-4593-88f4-067394d4ff8b" containerID="de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3" exitCode=137 Jul 22 16:43:08.914982 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.914873 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" Jul 22 16:43:08.914982 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.914887 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" event={"ID":"032e7114-a009-4593-88f4-067394d4ff8b","Type":"ContainerDied","Data":"205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd"} Jul 22 16:43:08.914982 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.914920 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" event={"ID":"032e7114-a009-4593-88f4-067394d4ff8b","Type":"ContainerDied","Data":"de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3"} Jul 22 16:43:08.914982 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.914939 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6" event={"ID":"032e7114-a009-4593-88f4-067394d4ff8b","Type":"ContainerDied","Data":"9b57c65acc4bb6af86b509d83d5088d4f0bca403062969a16623c88d2f37e631"} Jul 22 16:43:08.920307 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.920289 2565 scope.go:117] "RemoveContainer" containerID="d422d0af5d4d4ba53993ddaf3625de0444a0a5e65e3d7bae3f5d5b3095399d95" Jul 22 16:43:08.933848 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.933831 2565 scope.go:117] "RemoveContainer" containerID="fe42c6a79c3b0d980717bd2bc35b6723c74a9d3b5c41a8cc0c16db2b16ee19aa" Jul 22 16:43:08.940750 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.940734 2565 scope.go:117] "RemoveContainer" containerID="544bc3f932d9d7bee97ce7b3894ecda45a82ede98864eb40f511d3da0eb22bb3" Jul 22 16:43:08.947363 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.947343 2565 scope.go:117] "RemoveContainer" containerID="69012252362d857ae4aecc9da4c4aad8cec6e6f142c1597efd329d3ee1e0c339" Jul 22 16:43:08.954422 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.954365 2565 scope.go:117] "RemoveContainer" containerID="622512cacd4f9dc6a21407ae4dd7b1fa6a99dd43f4d132918382285afe214606" Jul 22 16:43:08.954812 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:43:08.954778 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"622512cacd4f9dc6a21407ae4dd7b1fa6a99dd43f4d132918382285afe214606\": container with ID starting with 622512cacd4f9dc6a21407ae4dd7b1fa6a99dd43f4d132918382285afe214606 not found: ID does not exist" containerID="622512cacd4f9dc6a21407ae4dd7b1fa6a99dd43f4d132918382285afe214606" Jul 22 16:43:08.954897 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.954811 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"622512cacd4f9dc6a21407ae4dd7b1fa6a99dd43f4d132918382285afe214606"} err="failed to get container status \"622512cacd4f9dc6a21407ae4dd7b1fa6a99dd43f4d132918382285afe214606\": rpc error: code = NotFound desc = could not find container \"622512cacd4f9dc6a21407ae4dd7b1fa6a99dd43f4d132918382285afe214606\": container with ID starting with 622512cacd4f9dc6a21407ae4dd7b1fa6a99dd43f4d132918382285afe214606 not found: ID does not exist" Jul 22 16:43:08.954897 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.954831 2565 scope.go:117] "RemoveContainer" containerID="d422d0af5d4d4ba53993ddaf3625de0444a0a5e65e3d7bae3f5d5b3095399d95" Jul 22 16:43:08.955163 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:43:08.955146 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d422d0af5d4d4ba53993ddaf3625de0444a0a5e65e3d7bae3f5d5b3095399d95\": container with ID starting with d422d0af5d4d4ba53993ddaf3625de0444a0a5e65e3d7bae3f5d5b3095399d95 not found: ID does not exist" containerID="d422d0af5d4d4ba53993ddaf3625de0444a0a5e65e3d7bae3f5d5b3095399d95" Jul 22 16:43:08.955243 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.955169 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d422d0af5d4d4ba53993ddaf3625de0444a0a5e65e3d7bae3f5d5b3095399d95"} err="failed to get container status \"d422d0af5d4d4ba53993ddaf3625de0444a0a5e65e3d7bae3f5d5b3095399d95\": rpc error: code = NotFound desc = could not find container \"d422d0af5d4d4ba53993ddaf3625de0444a0a5e65e3d7bae3f5d5b3095399d95\": container with ID starting with d422d0af5d4d4ba53993ddaf3625de0444a0a5e65e3d7bae3f5d5b3095399d95 not found: ID does not exist" Jul 22 16:43:08.955243 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.955186 2565 scope.go:117] "RemoveContainer" containerID="fe42c6a79c3b0d980717bd2bc35b6723c74a9d3b5c41a8cc0c16db2b16ee19aa" Jul 22 16:43:08.955465 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.955450 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv"] Jul 22 16:43:08.955512 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:43:08.955470 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe42c6a79c3b0d980717bd2bc35b6723c74a9d3b5c41a8cc0c16db2b16ee19aa\": container with ID starting with fe42c6a79c3b0d980717bd2bc35b6723c74a9d3b5c41a8cc0c16db2b16ee19aa not found: ID does not exist" containerID="fe42c6a79c3b0d980717bd2bc35b6723c74a9d3b5c41a8cc0c16db2b16ee19aa" Jul 22 16:43:08.955512 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.955492 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe42c6a79c3b0d980717bd2bc35b6723c74a9d3b5c41a8cc0c16db2b16ee19aa"} err="failed to get container status \"fe42c6a79c3b0d980717bd2bc35b6723c74a9d3b5c41a8cc0c16db2b16ee19aa\": rpc error: code = NotFound desc = could not find container \"fe42c6a79c3b0d980717bd2bc35b6723c74a9d3b5c41a8cc0c16db2b16ee19aa\": container with ID starting with fe42c6a79c3b0d980717bd2bc35b6723c74a9d3b5c41a8cc0c16db2b16ee19aa not found: ID does not exist" Jul 22 16:43:08.955512 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.955506 2565 scope.go:117] "RemoveContainer" containerID="544bc3f932d9d7bee97ce7b3894ecda45a82ede98864eb40f511d3da0eb22bb3" Jul 22 16:43:08.955782 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:43:08.955766 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"544bc3f932d9d7bee97ce7b3894ecda45a82ede98864eb40f511d3da0eb22bb3\": container with ID starting with 544bc3f932d9d7bee97ce7b3894ecda45a82ede98864eb40f511d3da0eb22bb3 not found: ID does not exist" containerID="544bc3f932d9d7bee97ce7b3894ecda45a82ede98864eb40f511d3da0eb22bb3" Jul 22 16:43:08.955833 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.955791 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"544bc3f932d9d7bee97ce7b3894ecda45a82ede98864eb40f511d3da0eb22bb3"} err="failed to get container status \"544bc3f932d9d7bee97ce7b3894ecda45a82ede98864eb40f511d3da0eb22bb3\": rpc error: code = NotFound desc = could not find container \"544bc3f932d9d7bee97ce7b3894ecda45a82ede98864eb40f511d3da0eb22bb3\": container with ID starting with 544bc3f932d9d7bee97ce7b3894ecda45a82ede98864eb40f511d3da0eb22bb3 not found: ID does not exist" Jul 22 16:43:08.955833 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.955810 2565 scope.go:117] "RemoveContainer" containerID="69012252362d857ae4aecc9da4c4aad8cec6e6f142c1597efd329d3ee1e0c339" Jul 22 16:43:08.956017 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:43:08.956002 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69012252362d857ae4aecc9da4c4aad8cec6e6f142c1597efd329d3ee1e0c339\": container with ID starting with 69012252362d857ae4aecc9da4c4aad8cec6e6f142c1597efd329d3ee1e0c339 not found: ID does not exist" containerID="69012252362d857ae4aecc9da4c4aad8cec6e6f142c1597efd329d3ee1e0c339" Jul 22 16:43:08.956062 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.956021 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69012252362d857ae4aecc9da4c4aad8cec6e6f142c1597efd329d3ee1e0c339"} err="failed to get container status \"69012252362d857ae4aecc9da4c4aad8cec6e6f142c1597efd329d3ee1e0c339\": rpc error: code = NotFound desc = could not find container \"69012252362d857ae4aecc9da4c4aad8cec6e6f142c1597efd329d3ee1e0c339\": container with ID starting with 69012252362d857ae4aecc9da4c4aad8cec6e6f142c1597efd329d3ee1e0c339 not found: ID does not exist" Jul 22 16:43:08.956062 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.956035 2565 scope.go:117] "RemoveContainer" containerID="097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d" Jul 22 16:43:08.958652 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.958633 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-db6955c6c-bhqrv"] Jul 22 16:43:08.963433 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.963419 2565 scope.go:117] "RemoveContainer" containerID="205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd" Jul 22 16:43:08.970374 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.970358 2565 scope.go:117] "RemoveContainer" containerID="317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c" Jul 22 16:43:08.975539 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.975521 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6"] Jul 22 16:43:08.977976 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.977907 2565 scope.go:117] "RemoveContainer" containerID="de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3" Jul 22 16:43:08.982836 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.982817 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-777d6cc998-nqpt6"] Jul 22 16:43:08.985024 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.985005 2565 scope.go:117] "RemoveContainer" containerID="bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f" Jul 22 16:43:08.991758 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.991742 2565 scope.go:117] "RemoveContainer" containerID="286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89" Jul 22 16:43:08.998209 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.998191 2565 scope.go:117] "RemoveContainer" containerID="097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d" Jul 22 16:43:08.998472 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:43:08.998448 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d\": container with ID starting with 097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d not found: ID does not exist" containerID="097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d" Jul 22 16:43:08.998546 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.998481 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d"} err="failed to get container status \"097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d\": rpc error: code = NotFound desc = could not find container \"097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d\": container with ID starting with 097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d not found: ID does not exist" Jul 22 16:43:08.998546 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.998501 2565 scope.go:117] "RemoveContainer" containerID="205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd" Jul 22 16:43:08.998747 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:43:08.998729 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd\": container with ID starting with 205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd not found: ID does not exist" containerID="205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd" Jul 22 16:43:08.998784 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.998752 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd"} err="failed to get container status \"205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd\": rpc error: code = NotFound desc = could not find container \"205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd\": container with ID starting with 205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd not found: ID does not exist" Jul 22 16:43:08.998784 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.998765 2565 scope.go:117] "RemoveContainer" containerID="317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c" Jul 22 16:43:08.998990 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:43:08.998974 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c\": container with ID starting with 317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c not found: ID does not exist" containerID="317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c" Jul 22 16:43:08.999050 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.998997 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c"} err="failed to get container status \"317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c\": rpc error: code = NotFound desc = could not find container \"317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c\": container with ID starting with 317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c not found: ID does not exist" Jul 22 16:43:08.999050 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.999020 2565 scope.go:117] "RemoveContainer" containerID="de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3" Jul 22 16:43:08.999223 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:43:08.999208 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3\": container with ID starting with de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3 not found: ID does not exist" containerID="de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3" Jul 22 16:43:08.999266 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.999227 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3"} err="failed to get container status \"de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3\": rpc error: code = NotFound desc = could not find container \"de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3\": container with ID starting with de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3 not found: ID does not exist" Jul 22 16:43:08.999266 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.999238 2565 scope.go:117] "RemoveContainer" containerID="bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f" Jul 22 16:43:08.999469 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:43:08.999451 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f\": container with ID starting with bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f not found: ID does not exist" containerID="bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f" Jul 22 16:43:08.999516 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.999474 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f"} err="failed to get container status \"bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f\": rpc error: code = NotFound desc = could not find container \"bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f\": container with ID starting with bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f not found: ID does not exist" Jul 22 16:43:08.999516 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.999488 2565 scope.go:117] "RemoveContainer" containerID="286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89" Jul 22 16:43:08.999690 ip-10-0-129-160 kubenswrapper[2565]: E0722 16:43:08.999674 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89\": container with ID starting with 286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89 not found: ID does not exist" containerID="286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89" Jul 22 16:43:08.999735 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.999693 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89"} err="failed to get container status \"286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89\": rpc error: code = NotFound desc = could not find container \"286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89\": container with ID starting with 286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89 not found: ID does not exist" Jul 22 16:43:08.999735 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.999705 2565 scope.go:117] "RemoveContainer" containerID="097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d" Jul 22 16:43:08.999887 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.999869 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d"} err="failed to get container status \"097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d\": rpc error: code = NotFound desc = could not find container \"097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d\": container with ID starting with 097f0f01a0f43ca45be2f26586f70aaa3bf7633acf79378b0286fdf1cd38b39d not found: ID does not exist" Jul 22 16:43:08.999932 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:08.999889 2565 scope.go:117] "RemoveContainer" containerID="205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd" Jul 22 16:43:09.000099 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:09.000080 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd"} err="failed to get container status \"205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd\": rpc error: code = NotFound desc = could not find container \"205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd\": container with ID starting with 205d669706a37d6a38c9fd5dfff57bb78b141b7db272176ac9e7ac00eaca42cd not found: ID does not exist" Jul 22 16:43:09.000172 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:09.000101 2565 scope.go:117] "RemoveContainer" containerID="317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c" Jul 22 16:43:09.000344 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:09.000326 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c"} err="failed to get container status \"317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c\": rpc error: code = NotFound desc = could not find container \"317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c\": container with ID starting with 317ec42ba2b860e23d072119d019f68899c11ae8b112701fd8624ccdec6a771c not found: ID does not exist" Jul 22 16:43:09.000344 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:09.000343 2565 scope.go:117] "RemoveContainer" containerID="de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3" Jul 22 16:43:09.000581 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:09.000566 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3"} err="failed to get container status \"de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3\": rpc error: code = NotFound desc = could not find container \"de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3\": container with ID starting with de16786b86a08997ab69629d3b685508dafd7292de7c8e869e7be83ce1eefde3 not found: ID does not exist" Jul 22 16:43:09.000628 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:09.000582 2565 scope.go:117] "RemoveContainer" containerID="bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f" Jul 22 16:43:09.000779 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:09.000761 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f"} err="failed to get container status \"bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f\": rpc error: code = NotFound desc = could not find container \"bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f\": container with ID starting with bec27e9a1791d498d89e804564cf8ef82c7cf7491bbb0d65d4e3a90caefc3c2f not found: ID does not exist" Jul 22 16:43:09.000826 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:09.000780 2565 scope.go:117] "RemoveContainer" containerID="286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89" Jul 22 16:43:09.000955 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:09.000936 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89"} err="failed to get container status \"286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89\": rpc error: code = NotFound desc = could not find container \"286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89\": container with ID starting with 286ce45e0af59a96726f01dc8e1bbf566b00704a4f6fc14577649e615b05fa89 not found: ID does not exist" Jul 22 16:43:09.526850 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:09.526826 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="032e7114-a009-4593-88f4-067394d4ff8b" path="/var/lib/kubelet/pods/032e7114-a009-4593-88f4-067394d4ff8b/volumes" Jul 22 16:43:09.527406 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:09.527376 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" path="/var/lib/kubelet/pods/5f078cf2-64b6-48dc-9884-bf41a9e035af/volumes" Jul 22 16:43:47.790623 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:47.790588 2565 ???:1] "http2: server: error reading preface from client 10.0.129.160:59752: read tcp 10.0.129.160:10250->10.0.129.160:59752: read: connection reset by peer" Jul 22 16:43:47.796540 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:47.796519 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jdctp_6b58efe5-eede-4fbf-a506-ae944342040f/global-pull-secret-syncer/0.log" Jul 22 16:43:47.981604 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:47.981581 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-q44t2_96df30a0-a2c9-42b3-9e2c-733801aa84b2/konnectivity-agent/0.log" Jul 22 16:43:48.087268 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:48.087219 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-160.ec2.internal_db69a2dc3be3a098e4e8d4e246ebb928/haproxy/0.log" Jul 22 16:43:52.050213 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.050132 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-994875d55-g2w9s_dd837a3a-0d04-4cb7-9812-b0f1e306a3b3/metrics-server/0.log" Jul 22 16:43:52.218058 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.218031 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5shsb_588ab578-e02f-44ab-98c5-5ca5b4b9e35c/node-exporter/0.log" Jul 22 16:43:52.244741 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.244714 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5shsb_588ab578-e02f-44ab-98c5-5ca5b4b9e35c/kube-rbac-proxy/0.log" Jul 22 16:43:52.275937 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.275918 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5shsb_588ab578-e02f-44ab-98c5-5ca5b4b9e35c/init-textfile/0.log" Jul 22 16:43:52.402528 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.402506 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-t7p28_e212f29c-872c-45ec-a580-73ac0e757d9b/kube-rbac-proxy-main/0.log" Jul 22 16:43:52.430999 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.430980 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-t7p28_e212f29c-872c-45ec-a580-73ac0e757d9b/kube-rbac-proxy-self/0.log" Jul 22 16:43:52.461593 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.461573 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-t7p28_e212f29c-872c-45ec-a580-73ac0e757d9b/openshift-state-metrics/0.log" Jul 22 16:43:52.501090 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.501071 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6f5cc774-8c4e-43fc-a268-8b99de6e102e/prometheus/0.log" Jul 22 16:43:52.531016 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.531001 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6f5cc774-8c4e-43fc-a268-8b99de6e102e/config-reloader/0.log" Jul 22 16:43:52.557698 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.557676 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6f5cc774-8c4e-43fc-a268-8b99de6e102e/thanos-sidecar/0.log" Jul 22 16:43:52.585519 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.585500 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6f5cc774-8c4e-43fc-a268-8b99de6e102e/kube-rbac-proxy-web/0.log" Jul 22 16:43:52.618244 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.618226 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6f5cc774-8c4e-43fc-a268-8b99de6e102e/kube-rbac-proxy/0.log" Jul 22 16:43:52.656797 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.656741 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6f5cc774-8c4e-43fc-a268-8b99de6e102e/kube-rbac-proxy-thanos/0.log" Jul 22 16:43:52.691602 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.691588 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6f5cc774-8c4e-43fc-a268-8b99de6e102e/init-config-reloader/0.log" Jul 22 16:43:52.730703 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.730680 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-jvbvj_ae4ea54d-b612-480c-8415-c13e53c2588b/prometheus-operator/0.log" Jul 22 16:43:52.759845 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.759830 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-jvbvj_ae4ea54d-b612-480c-8415-c13e53c2588b/kube-rbac-proxy/0.log" Jul 22 16:43:52.803603 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:52.803586 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-dpt7j_738e099a-c837-4176-9a84-04e5414a2959/prometheus-operator-admission-webhook/0.log" Jul 22 16:43:54.236963 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:54.236940 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-5hjsz_cf370697-0317-4d1d-99c3-f09c1fc2ed4a/networking-console-plugin/0.log" Jul 22 16:43:55.258070 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258040 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k"] Jul 22 16:43:55.258452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258299 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="feast-apply" Jul 22 16:43:55.258452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258313 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="feast-apply" Jul 22 16:43:55.258452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258336 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="online" Jul 22 16:43:55.258452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258344 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="online" Jul 22 16:43:55.258452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258356 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="offline" Jul 22 16:43:55.258452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258365 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="offline" Jul 22 16:43:55.258452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258375 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="feast-init" Jul 22 16:43:55.258452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258383 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="feast-init" Jul 22 16:43:55.258452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258408 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="ui" Jul 22 16:43:55.258452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258417 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="ui" Jul 22 16:43:55.258452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258429 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="feast-init" Jul 22 16:43:55.258452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258438 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="feast-init" Jul 22 16:43:55.258452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258455 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="ui" Jul 22 16:43:55.258452 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258461 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="ui" Jul 22 16:43:55.259156 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258472 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="offline" Jul 22 16:43:55.259156 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258481 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="offline" Jul 22 16:43:55.259156 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258490 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="registry" Jul 22 16:43:55.259156 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258495 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="registry" Jul 22 16:43:55.259156 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258506 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="feast-apply" Jul 22 16:43:55.259156 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258516 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="feast-apply" Jul 22 16:43:55.259156 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258526 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="online" Jul 22 16:43:55.259156 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258535 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="online" Jul 22 16:43:55.259156 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258623 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="online" Jul 22 16:43:55.259156 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258635 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="ui" Jul 22 16:43:55.259156 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258645 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="offline" Jul 22 16:43:55.259156 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258652 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="online" Jul 22 16:43:55.259156 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258661 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f078cf2-64b6-48dc-9884-bf41a9e035af" containerName="offline" Jul 22 16:43:55.259156 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258669 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="registry" Jul 22 16:43:55.259156 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.258677 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="032e7114-a009-4593-88f4-067394d4ff8b" containerName="ui" Jul 22 16:43:55.261179 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.261163 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.264136 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.264118 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wwm22\"/\"openshift-service-ca.crt\"" Jul 22 16:43:55.264241 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.264142 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wwm22\"/\"default-dockercfg-9z286\"" Jul 22 16:43:55.265166 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.265150 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wwm22\"/\"kube-root-ca.crt\"" Jul 22 16:43:55.270335 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.270311 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k"] Jul 22 16:43:55.362861 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.362843 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/828047f7-6af4-4927-a959-8cb78d18b6b8-podres\") pod \"perf-node-gather-daemonset-nnq8k\" (UID: \"828047f7-6af4-4927-a959-8cb78d18b6b8\") " pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.362950 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.362878 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtc49\" (UniqueName: \"kubernetes.io/projected/828047f7-6af4-4927-a959-8cb78d18b6b8-kube-api-access-vtc49\") pod \"perf-node-gather-daemonset-nnq8k\" (UID: \"828047f7-6af4-4927-a959-8cb78d18b6b8\") " pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.362950 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.362922 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/828047f7-6af4-4927-a959-8cb78d18b6b8-sys\") pod \"perf-node-gather-daemonset-nnq8k\" (UID: \"828047f7-6af4-4927-a959-8cb78d18b6b8\") " pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.362950 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.362940 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/828047f7-6af4-4927-a959-8cb78d18b6b8-lib-modules\") pod \"perf-node-gather-daemonset-nnq8k\" (UID: \"828047f7-6af4-4927-a959-8cb78d18b6b8\") " pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.363058 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.362958 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/828047f7-6af4-4927-a959-8cb78d18b6b8-proc\") pod \"perf-node-gather-daemonset-nnq8k\" (UID: \"828047f7-6af4-4927-a959-8cb78d18b6b8\") " pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.464145 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.464123 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/828047f7-6af4-4927-a959-8cb78d18b6b8-podres\") pod \"perf-node-gather-daemonset-nnq8k\" (UID: \"828047f7-6af4-4927-a959-8cb78d18b6b8\") " pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.464218 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.464159 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtc49\" (UniqueName: \"kubernetes.io/projected/828047f7-6af4-4927-a959-8cb78d18b6b8-kube-api-access-vtc49\") pod \"perf-node-gather-daemonset-nnq8k\" (UID: \"828047f7-6af4-4927-a959-8cb78d18b6b8\") " pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.464218 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.464179 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/828047f7-6af4-4927-a959-8cb78d18b6b8-sys\") pod \"perf-node-gather-daemonset-nnq8k\" (UID: \"828047f7-6af4-4927-a959-8cb78d18b6b8\") " pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.464218 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.464197 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/828047f7-6af4-4927-a959-8cb78d18b6b8-lib-modules\") pod \"perf-node-gather-daemonset-nnq8k\" (UID: \"828047f7-6af4-4927-a959-8cb78d18b6b8\") " pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.464315 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.464253 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/828047f7-6af4-4927-a959-8cb78d18b6b8-proc\") pod \"perf-node-gather-daemonset-nnq8k\" (UID: \"828047f7-6af4-4927-a959-8cb78d18b6b8\") " pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.464315 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.464283 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/828047f7-6af4-4927-a959-8cb78d18b6b8-podres\") pod \"perf-node-gather-daemonset-nnq8k\" (UID: \"828047f7-6af4-4927-a959-8cb78d18b6b8\") " pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.464315 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.464295 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/828047f7-6af4-4927-a959-8cb78d18b6b8-proc\") pod \"perf-node-gather-daemonset-nnq8k\" (UID: \"828047f7-6af4-4927-a959-8cb78d18b6b8\") " pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.464315 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.464261 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/828047f7-6af4-4927-a959-8cb78d18b6b8-sys\") pod \"perf-node-gather-daemonset-nnq8k\" (UID: \"828047f7-6af4-4927-a959-8cb78d18b6b8\") " pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.464458 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.464337 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/828047f7-6af4-4927-a959-8cb78d18b6b8-lib-modules\") pod \"perf-node-gather-daemonset-nnq8k\" (UID: \"828047f7-6af4-4927-a959-8cb78d18b6b8\") " pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.474672 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.474654 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtc49\" (UniqueName: \"kubernetes.io/projected/828047f7-6af4-4927-a959-8cb78d18b6b8-kube-api-access-vtc49\") pod \"perf-node-gather-daemonset-nnq8k\" (UID: \"828047f7-6af4-4927-a959-8cb78d18b6b8\") " pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.571373 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.571325 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:55.697060 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:55.697007 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k"] Jul 22 16:43:55.703558 ip-10-0-129-160 kubenswrapper[2565]: W0722 16:43:55.703532 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod828047f7_6af4_4927_a959_8cb78d18b6b8.slice/crio-fa06777c741df48be06f041ff183bd5f8c31d4d461f2e7a592565e79d331755e WatchSource:0}: Error finding container fa06777c741df48be06f041ff183bd5f8c31d4d461f2e7a592565e79d331755e: Status 404 returned error can't find the container with id fa06777c741df48be06f041ff183bd5f8c31d4d461f2e7a592565e79d331755e Jul 22 16:43:56.088686 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:56.088660 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" event={"ID":"828047f7-6af4-4927-a959-8cb78d18b6b8","Type":"ContainerStarted","Data":"87a03cb186bcc29ae1d98d8e980301d03f8e35028f65d2e3aa16768ec30c28ba"} Jul 22 16:43:56.088798 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:56.088704 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" event={"ID":"828047f7-6af4-4927-a959-8cb78d18b6b8","Type":"ContainerStarted","Data":"fa06777c741df48be06f041ff183bd5f8c31d4d461f2e7a592565e79d331755e"} Jul 22 16:43:56.088846 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:56.088797 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:43:56.108204 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:56.108164 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" podStartSLOduration=1.108150169 podStartE2EDuration="1.108150169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 16:43:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 16:43:56.106435751 +0000 UTC m=+1933.119602791" watchObservedRunningTime="2026-07-22 16:43:56.108150169 +0000 UTC m=+1933.121317206" Jul 22 16:43:56.568281 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:56.568256 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m9wx5_4c228a9d-7811-459b-8b7a-6dd6e6f5a76a/dns/0.log" Jul 22 16:43:56.598594 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:56.598580 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m9wx5_4c228a9d-7811-459b-8b7a-6dd6e6f5a76a/kube-rbac-proxy/0.log" Jul 22 16:43:56.711569 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:56.711550 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-st8vx_6aae32f3-293e-4611-b044-892e51135c16/dns-node-resolver/0.log" Jul 22 16:43:57.232252 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:57.232229 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bpcxs_3564cf28-82d3-4856-8ee0-17c61398d176/node-ca/0.log" Jul 22 16:43:58.361485 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:58.361451 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-57gnc_790a8029-54c9-4de7-b572-10c17aae7e42/serve-healthcheck-canary/0.log" Jul 22 16:43:58.986816 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:58.986795 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mgbrd_4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c/kube-rbac-proxy/0.log" Jul 22 16:43:59.017812 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:59.017788 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mgbrd_4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c/exporter/0.log" Jul 22 16:43:59.049588 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:43:59.049566 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mgbrd_4d7af12d-36fa-4ff9-b6d9-0dbae4e0bc3c/extractor/0.log" Jul 22 16:44:02.101981 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:02.101957 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wwm22/perf-node-gather-daemonset-nnq8k" Jul 22 16:44:03.875868 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:03.875741 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-nzb4f_592cc5de-ba98-448f-9120-4690a117a4ca/kube-storage-version-migrator-operator/1.log" Jul 22 16:44:03.876661 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:03.876642 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-nzb4f_592cc5de-ba98-448f-9120-4690a117a4ca/kube-storage-version-migrator-operator/0.log" Jul 22 16:44:05.128313 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:05.128288 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bj4s6_592b0f74-1d67-4361-85ba-6fe61394a87e/kube-multus-additional-cni-plugins/0.log" Jul 22 16:44:05.155238 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:05.155219 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bj4s6_592b0f74-1d67-4361-85ba-6fe61394a87e/egress-router-binary-copy/0.log" Jul 22 16:44:05.180335 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:05.180317 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bj4s6_592b0f74-1d67-4361-85ba-6fe61394a87e/cni-plugins/0.log" Jul 22 16:44:05.209826 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:05.209806 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bj4s6_592b0f74-1d67-4361-85ba-6fe61394a87e/bond-cni-plugin/0.log" Jul 22 16:44:05.236305 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:05.236287 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bj4s6_592b0f74-1d67-4361-85ba-6fe61394a87e/routeoverride-cni/0.log" Jul 22 16:44:05.261855 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:05.261834 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bj4s6_592b0f74-1d67-4361-85ba-6fe61394a87e/whereabouts-cni-bincopy/0.log" Jul 22 16:44:05.287203 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:05.287186 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bj4s6_592b0f74-1d67-4361-85ba-6fe61394a87e/whereabouts-cni/0.log" Jul 22 16:44:05.556493 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:05.556441 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vwml7_9fafd1b8-f815-4a95-982f-fe79e9086034/kube-multus/0.log" Jul 22 16:44:05.581041 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:05.581019 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gkw9w_3dc7600c-4705-48d3-a410-eebfaa40c9cf/network-metrics-daemon/0.log" Jul 22 16:44:05.606633 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:05.606612 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gkw9w_3dc7600c-4705-48d3-a410-eebfaa40c9cf/kube-rbac-proxy/0.log" Jul 22 16:44:07.027961 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:07.027931 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xjtvj_29719853-4c4b-4f4f-bcc3-2cc539e6da52/ovn-controller/0.log" Jul 22 16:44:07.049518 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:07.049495 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xjtvj_29719853-4c4b-4f4f-bcc3-2cc539e6da52/ovn-acl-logging/0.log" Jul 22 16:44:07.079180 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:07.079157 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xjtvj_29719853-4c4b-4f4f-bcc3-2cc539e6da52/kube-rbac-proxy-node/0.log" Jul 22 16:44:07.104556 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:07.104538 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xjtvj_29719853-4c4b-4f4f-bcc3-2cc539e6da52/kube-rbac-proxy-ovn-metrics/0.log" Jul 22 16:44:07.127027 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:07.127003 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xjtvj_29719853-4c4b-4f4f-bcc3-2cc539e6da52/northd/0.log" Jul 22 16:44:07.151942 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:07.151922 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xjtvj_29719853-4c4b-4f4f-bcc3-2cc539e6da52/nbdb/0.log" Jul 22 16:44:07.175807 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:07.175789 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xjtvj_29719853-4c4b-4f4f-bcc3-2cc539e6da52/sbdb/0.log" Jul 22 16:44:07.218758 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:07.218723 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xjtvj_29719853-4c4b-4f4f-bcc3-2cc539e6da52/ovnkube-controller/0.log" Jul 22 16:44:08.282287 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:08.282265 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-blvd8_7d94b09b-161f-4498-b0ac-9f3c0b104444/network-check-target-container/0.log" Jul 22 16:44:09.352136 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:09.352108 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-z9cck_0097d54d-435a-471d-93f0-e9075cdec91c/iptables-alerter/0.log" Jul 22 16:44:09.981535 ip-10-0-129-160 kubenswrapper[2565]: I0722 16:44:09.981513 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-6lbnb_536aeccf-be47-44e1-a65c-572c81ba240a/tuned/0.log"