Jul 28 12:04:02.823020 ip-10-0-129-158 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 28 12:04:02.823031 ip-10-0-129-158 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 28 12:04:02.823039 ip-10-0-129-158 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 28 12:04:02.823253 ip-10-0-129-158 systemd[1]: Failed to start Kubernetes Kubelet. Jul 28 12:04:13.021684 ip-10-0-129-158 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 28 12:04:13.021705 ip-10-0-129-158 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 0674d3e6376b420dbde7ed7a993f5a95 -- Jul 28 12:06:05.548616 ip-10-0-129-158 systemd[1]: Starting Kubernetes Kubelet... Jul 28 12:06:05.969847 ip-10-0-129-158 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 12:06:05.969847 ip-10-0-129-158 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 28 12:06:05.969847 ip-10-0-129-158 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 12:06:05.969847 ip-10-0-129-158 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 28 12:06:05.969847 ip-10-0-129-158 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 12:06:05.970822 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.970732 2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 28 12:06:05.976093 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976068 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 12:06:05.976093 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976089 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 12:06:05.976093 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976093 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 12:06:05.976093 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976096 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 12:06:05.976093 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976099 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 12:06:05.976093 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976102 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 12:06:05.976093 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976105 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976108 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976111 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976114 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976117 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976120 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976122 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976125 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976127 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976130 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976132 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976135 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976137 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976140 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976143 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976145 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976148 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976150 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976152 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976155 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 12:06:05.976361 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976158 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976161 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976164 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976166 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976169 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976171 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976181 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976184 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976186 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976189 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976191 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976194 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976196 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976199 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976202 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976204 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976207 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976209 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976212 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976214 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 12:06:05.976837 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976217 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976219 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976222 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976224 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976227 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976229 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976232 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976234 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976237 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976240 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976242 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976245 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976247 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976250 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976252 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976255 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976257 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976261 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976264 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976268 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 12:06:05.977339 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976273 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976276 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976280 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976283 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976288 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976292 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976296 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976299 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976317 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976320 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976323 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976326 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976329 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976331 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976334 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976336 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976339 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976349 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976351 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 12:06:05.977820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976354 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976356 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976359 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976362 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976365 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976367 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976370 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976373 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976375 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976378 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976380 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976383 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976386 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.976388 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977091 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977097 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977100 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977103 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977105 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 12:06:05.978269 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977108 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977111 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977114 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977117 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977120 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977122 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977125 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977128 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977131 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977133 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977136 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977139 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977142 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977144 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977148 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977150 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977152 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977155 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977157 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977160 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 12:06:05.978744 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977162 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977165 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977167 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977170 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977172 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977175 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977179 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977183 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977185 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977188 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977190 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977193 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977197 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977199 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977202 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977204 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977207 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977210 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977212 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 12:06:05.979247 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977215 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977218 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977220 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977223 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977225 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977227 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977231 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977233 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977236 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977239 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977242 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977245 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977250 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977253 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977255 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977258 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977260 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977263 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977266 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977268 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 12:06:05.979735 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977271 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977273 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977276 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977279 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977281 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977284 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977287 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977289 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977292 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977294 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977297 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977299 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977317 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977320 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977322 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977327 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977330 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977333 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977336 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 12:06:05.980217 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977339 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977342 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977345 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977348 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977350 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977353 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977356 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977358 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977361 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977363 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977366 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977368 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977371 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977373 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977376 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977378 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977747 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977752 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977756 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977759 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 12:06:05.980711 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977762 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977764 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977767 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977770 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977772 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977775 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977778 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977780 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977783 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977786 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977789 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977791 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977795 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977798 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977801 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977803 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977807 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977811 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977814 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977817 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 12:06:05.981183 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977820 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977823 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977826 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977829 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977832 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977835 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977837 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977840 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977842 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977845 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977848 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977850 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977853 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977855 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977858 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977861 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977863 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977866 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977869 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 12:06:05.981675 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977872 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977874 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977876 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977879 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977882 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977884 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977887 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977890 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977892 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977895 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977897 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977900 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977902 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977905 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977907 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977910 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977912 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977915 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977917 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977920 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 12:06:05.982167 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977922 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977924 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977927 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977930 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977933 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977936 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977939 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977942 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977945 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977947 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977950 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977953 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977961 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977964 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977968 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977972 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977975 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977978 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977980 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 12:06:05.982663 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977983 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977986 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977989 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977991 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977994 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977996 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.977999 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978001 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978003 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978006 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978008 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978011 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978013 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978016 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978018 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978021 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978026 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978098 2575 flags.go:64] FLAG: --address="0.0.0.0" Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978106 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978113 2575 flags.go:64] FLAG: --anonymous-auth="true" Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978118 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 28 12:06:05.983123 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978122 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978125 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978130 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978136 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978139 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978142 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978145 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978149 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978152 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978155 2575 flags.go:64] FLAG: --cgroup-root="" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978157 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978160 2575 flags.go:64] FLAG: --client-ca-file="" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978164 2575 flags.go:64] FLAG: --cloud-provider="external" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978167 2575 flags.go:64] FLAG: --cluster-dns="[]" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978171 2575 flags.go:64] FLAG: --cluster-domain="" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978173 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978177 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978180 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978183 2575 flags.go:64] FLAG: --container-log-max-files="5" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978187 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978189 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978192 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978196 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978199 2575 flags.go:64] FLAG: --contention-profiling="false" Jul 28 12:06:05.983671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978202 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978204 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978208 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978211 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978216 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978219 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978222 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978224 2575 flags.go:64] FLAG: --enable-load-reader="false" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978227 2575 flags.go:64] FLAG: --enable-server="true" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978230 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978235 2575 flags.go:64] FLAG: --event-burst="100" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978240 2575 flags.go:64] FLAG: --event-qps="50" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978242 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978245 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978249 2575 flags.go:64] FLAG: --eviction-hard="" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978253 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978256 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978259 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978262 2575 flags.go:64] FLAG: --eviction-soft="" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978265 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978268 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978271 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978274 2575 flags.go:64] FLAG: --experimental-mounter-path="" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978277 2575 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978280 2575 flags.go:64] FLAG: --fail-swap-on="true" Jul 28 12:06:05.984240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978283 2575 flags.go:64] FLAG: --feature-gates="" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978287 2575 flags.go:64] FLAG: --file-check-frequency="20s" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978290 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978294 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978297 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978300 2575 flags.go:64] FLAG: --healthz-port="10248" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978316 2575 flags.go:64] FLAG: --help="false" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978319 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-129-158.ec2.internal" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978322 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978325 2575 flags.go:64] FLAG: --http-check-frequency="20s" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978329 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978332 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978335 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978338 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978341 2575 flags.go:64] FLAG: --image-service-endpoint="" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978344 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978346 2575 flags.go:64] FLAG: --kube-api-burst="100" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978349 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978353 2575 flags.go:64] FLAG: --kube-api-qps="50" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978356 2575 flags.go:64] FLAG: --kube-reserved="" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978359 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978362 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978365 2575 flags.go:64] FLAG: --kubelet-cgroups="" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978367 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 28 12:06:05.984883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978370 2575 flags.go:64] FLAG: --lock-file="" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978373 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978376 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978378 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978384 2575 flags.go:64] FLAG: --log-json-split-stream="false" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978387 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978391 2575 flags.go:64] FLAG: --log-text-split-stream="false" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978393 2575 flags.go:64] FLAG: --logging-format="text" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978396 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978400 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978402 2575 flags.go:64] FLAG: --manifest-url="" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978405 2575 flags.go:64] FLAG: --manifest-url-header="" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978409 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978412 2575 flags.go:64] FLAG: --max-open-files="1000000" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978416 2575 flags.go:64] FLAG: --max-pods="110" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978419 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978422 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978425 2575 flags.go:64] FLAG: --memory-manager-policy="None" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978428 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978431 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978434 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978437 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978444 2575 flags.go:64] FLAG: --node-status-max-images="50" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978447 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978450 2575 flags.go:64] FLAG: --oom-score-adj="-999" Jul 28 12:06:05.985595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978453 2575 flags.go:64] FLAG: --pod-cidr="" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978457 2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978462 2575 flags.go:64] FLAG: --pod-manifest-path="" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978465 2575 flags.go:64] FLAG: --pod-max-pids="-1" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978468 2575 flags.go:64] FLAG: --pods-per-core="0" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978471 2575 flags.go:64] FLAG: --port="10250" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978474 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978477 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06ee490cb3ccf3d40" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978480 2575 flags.go:64] FLAG: --qos-reserved="" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978483 2575 flags.go:64] FLAG: --read-only-port="10255" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978486 2575 flags.go:64] FLAG: --register-node="true" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978489 2575 flags.go:64] FLAG: --register-with-taints="" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978492 2575 flags.go:64] FLAG: --registry-burst="10" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978496 2575 flags.go:64] FLAG: --registry-qps="5" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978499 2575 flags.go:64] FLAG: --reserved-cpus="" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978502 2575 flags.go:64] FLAG: --reserved-memory="" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978505 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978508 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978511 2575 flags.go:64] FLAG: --rotate-certificates="false" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978514 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978517 2575 flags.go:64] FLAG: --runonce="false" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978520 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978523 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978526 2575 flags.go:64] FLAG: --seccomp-default="false" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978529 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 28 12:06:05.986192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978531 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978534 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978537 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978540 2575 flags.go:64] FLAG: --storage-driver-password="root" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978543 2575 flags.go:64] FLAG: --storage-driver-secure="false" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978546 2575 flags.go:64] FLAG: --storage-driver-table="stats" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978549 2575 flags.go:64] FLAG: --storage-driver-user="root" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978552 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978556 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978559 2575 flags.go:64] FLAG: --system-cgroups="" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978561 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978566 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978569 2575 flags.go:64] FLAG: --tls-cert-file="" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978572 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978576 2575 flags.go:64] FLAG: --tls-min-version="" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978578 2575 flags.go:64] FLAG: --tls-private-key-file="" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978581 2575 flags.go:64] FLAG: --topology-manager-policy="none" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978584 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978587 2575 flags.go:64] FLAG: --topology-manager-scope="container" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978589 2575 flags.go:64] FLAG: --v="2" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978593 2575 flags.go:64] FLAG: --version="false" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978598 2575 flags.go:64] FLAG: --vmodule="" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978602 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.978605 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978717 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 12:06:05.986828 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978721 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978724 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978727 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978729 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978732 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978735 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978738 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978740 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978743 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978745 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978748 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978751 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978754 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978756 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978759 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978763 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978765 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978768 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978770 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978773 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 12:06:05.987478 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978775 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978778 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978780 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978783 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978785 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978788 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978792 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978795 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978798 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978801 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978804 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978806 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978809 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978811 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978813 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978816 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978818 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978821 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978824 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 12:06:05.987961 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978826 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978830 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978833 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978837 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978841 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978844 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978847 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978850 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978856 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978858 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978861 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978864 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978867 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978869 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978872 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978875 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978877 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978880 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978883 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 12:06:05.988438 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978885 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978888 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978890 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978893 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978896 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978899 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978902 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978904 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978907 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978909 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978912 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978915 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978918 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978920 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978924 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978926 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978929 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978932 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978935 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978938 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 12:06:05.988892 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978941 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978945 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978947 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978951 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978953 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978955 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978958 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978960 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978963 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978966 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978968 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978971 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978974 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978976 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978979 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978981 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978984 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978987 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978990 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 12:06:05.989378 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.978992 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 12:06:05.989829 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.979622 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 12:06:05.989829 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.986357 2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 28 12:06:05.989829 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.986375 2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 28 12:06:05.989829 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.986395 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 28 12:06:05.989829 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.986401 2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 28 12:06:05.989829 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986457 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 12:06:05.989829 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986462 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 12:06:05.989829 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986465 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 12:06:05.989829 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986468 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 12:06:05.989829 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986471 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 12:06:05.989829 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986474 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 12:06:05.989829 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986477 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 12:06:05.989829 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986479 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 12:06:05.989829 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986482 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986485 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986488 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986491 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986493 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986496 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986499 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986502 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986504 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986507 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986510 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986513 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986515 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986518 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986521 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986523 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986526 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986528 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986531 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986533 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 12:06:05.990164 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986536 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986540 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986543 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986545 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986548 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986550 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986553 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986555 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986558 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986561 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986563 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986566 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986568 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986571 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986573 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986576 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986578 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986582 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986586 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 12:06:05.990664 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986589 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986592 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986594 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986597 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986600 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986603 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986605 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986608 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986610 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986613 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986616 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986618 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986621 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986623 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986627 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986630 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986632 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986635 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986639 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986642 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 12:06:05.991118 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986645 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986647 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986650 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986652 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986655 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986657 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986660 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986663 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986665 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986668 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986670 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986673 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986675 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986678 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986681 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986683 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986686 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986688 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986691 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986693 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 12:06:05.991623 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986695 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 12:06:05.992105 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986698 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 12:06:05.992105 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986701 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 12:06:05.992105 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986703 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 12:06:05.992105 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986706 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 12:06:05.992105 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986710 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 12:06:05.992105 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986713 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 12:06:05.992105 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986716 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 12:06:05.992105 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986720 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 12:06:05.992105 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986723 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 12:06:05.992105 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986726 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 12:06:05.992105 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986728 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 12:06:05.992105 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.986733 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 12:06:05.992105 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986842 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 12:06:05.992105 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986846 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 12:06:05.992105 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986849 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986851 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986854 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986856 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986859 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986862 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986864 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986867 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986869 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986871 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986874 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986877 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986881 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986884 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986886 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986889 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986891 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986894 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986896 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 12:06:05.992561 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986899 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986901 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986904 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986907 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986910 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986913 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986915 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986918 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986921 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986924 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986926 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986929 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986931 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986934 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986936 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986939 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986941 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986943 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986946 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986948 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 12:06:05.993014 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986951 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986953 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986955 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986958 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986960 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986963 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986966 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986968 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986970 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986973 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986975 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986978 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986980 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986982 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986985 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986988 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986990 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986993 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986995 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 12:06:05.993511 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.986998 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987001 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987003 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987006 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987008 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987010 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987013 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987016 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987018 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987020 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987023 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987025 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987028 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987030 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987032 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987035 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987037 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987040 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987042 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987045 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 12:06:05.994141 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987047 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987051 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987055 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987057 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987060 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987063 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987066 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987069 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987071 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987074 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987076 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987079 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987081 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987084 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987087 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987089 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987092 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987094 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 12:06:05.994976 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:05.987098 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 12:06:05.995550 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.987103 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 12:06:05.995550 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.987204 2575 server.go:962] "Client rotation is on, will bootstrap in background" Jul 28 12:06:05.995550 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.989279 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 28 12:06:05.995550 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.990775 2575 server.go:1019] "Starting client certificate rotation" Jul 28 12:06:05.995550 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.990868 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 12:06:05.995550 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.990908 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 12:06:05.995550 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.993124 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 12:06:05.999155 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:05.999136 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 12:06:06.011405 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.011386 2575 log.go:25] "Validated CRI v1 runtime API" Jul 28 12:06:06.013479 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.013466 2575 log.go:25] "Validated CRI v1 image API" Jul 28 12:06:06.015505 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.015488 2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 28 12:06:06.019778 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.019758 2575 fs.go:135] Filesystem UUIDs: map[6968e8b2-102d-45b1-8c4e-7bb0f1686e4f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f6f3add1-bbf2-4b69-9cb0-c1ec7e521492:/dev/nvme0n1p3] Jul 28 12:06:06.019840 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.019776 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 28 12:06:06.021729 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.021711 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 12:06:06.024837 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.024733 2575 manager.go:216] Machine: {Timestamp:2026-07-28 12:06:06.023431694 +0000 UTC m=+0.348096962 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100834 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2578df53878bd9763c2b08c585dc42 SystemUUID:ec2578df-5387-8bd9-763c-2b08c585dc42 BootID:0674d3e6-376b-420d-bde7-ed7a993f5a95 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5a:8b:24:ba:79 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5a:8b:24:ba:79 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5e:d2:d1:22:70:12 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 28 12:06:06.024837 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.024828 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 28 12:06:06.024958 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.024908 2575 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 28 12:06:06.028455 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.028432 2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 28 12:06:06.028592 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.028458 2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-158.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 28 12:06:06.029252 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.029242 2575 topology_manager.go:138] "Creating topology manager with none policy" Jul 28 12:06:06.029283 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.029262 2575 container_manager_linux.go:309] "Creating device plugin manager" Jul 28 12:06:06.029283 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.029270 2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 12:06:06.029933 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.029923 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 12:06:06.030722 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.030712 2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 28 12:06:06.032533 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.032522 2575 state_mem.go:36] "Initialized new in-memory state store" Jul 28 12:06:06.032656 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.032646 2575 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 28 12:06:06.032691 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.032679 2575 kubelet.go:486] "Attempting to sync node with API server" Jul 28 12:06:06.032721 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.032692 2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 28 12:06:06.032721 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.032712 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 28 12:06:06.032771 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.032725 2575 kubelet.go:398] "Adding apiserver pod source" Jul 28 12:06:06.032771 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.032736 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 28 12:06:06.033155 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.033142 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 12:06:06.033215 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.033160 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 12:06:06.035505 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.035486 2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 28 12:06:06.037749 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.037735 2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 28 12:06:06.037838 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.037765 2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 28 12:06:06.039666 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.039653 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 28 12:06:06.039723 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.039670 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 28 12:06:06.039723 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.039677 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 28 12:06:06.039723 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.039683 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 28 12:06:06.039723 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.039703 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 28 12:06:06.039723 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.039709 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 28 12:06:06.039723 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.039718 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 28 12:06:06.039723 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.039723 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 28 12:06:06.039948 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.039729 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 28 12:06:06.039948 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.039735 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 28 12:06:06.039948 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.039748 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 28 12:06:06.039948 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.039754 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 28 12:06:06.041570 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.041559 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 28 12:06:06.041570 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.041568 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 28 12:06:06.044936 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.044921 2575 server.go:1268] "Started kubelet" Jul 28 12:06:06.045043 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.045014 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 28 12:06:06.045100 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.045037 2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 28 12:06:06.045100 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.045062 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 28 12:06:06.048538 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.048512 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lwscs" Jul 28 12:06:06.048627 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.048579 2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 28 12:06:06.048594 ip-10-0-129-158 systemd[1]: Started Kubernetes Kubelet. Jul 28 12:06:06.050246 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.050227 2575 server.go:310] "Adding debug handlers to kubelet server" Jul 28 12:06:06.052823 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.052796 2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 28 12:06:06.053702 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.053682 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-158.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 28 12:06:06.053702 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.053681 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 28 12:06:06.053805 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.053719 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-158.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 28 12:06:06.056537 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.055517 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-158.ec2.internal.18c672a780c7cc00 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-158.ec2.internal,UID:ip-10-0-129-158.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-158.ec2.internal,},FirstTimestamp:2026-07-28 12:06:06.044662784 +0000 UTC m=+0.369328053,LastTimestamp:2026-07-28 12:06:06.044662784 +0000 UTC m=+0.369328053,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-158.ec2.internal,}" Jul 28 12:06:06.059704 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.059541 2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 28 12:06:06.059704 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.059558 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 28 12:06:06.059704 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.059634 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-158.ec2.internal\" not found" Jul 28 12:06:06.059972 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.059955 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lwscs" Jul 28 12:06:06.060320 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.060292 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 28 12:06:06.060399 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.060356 2575 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 28 12:06:06.060399 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.060368 2575 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 28 12:06:06.060505 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.060473 2575 reconstruct.go:97] "Volume reconstruction finished" Jul 28 12:06:06.060505 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.060484 2575 reconciler.go:29] "Reconciler: start to sync state" Jul 28 12:06:06.060699 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.060686 2575 factory.go:55] Registering systemd factory Jul 28 12:06:06.060752 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.060727 2575 factory.go:223] Registration of the systemd container factory successfully Jul 28 12:06:06.061053 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.061038 2575 factory.go:153] Registering CRI-O factory Jul 28 12:06:06.061053 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.061052 2575 factory.go:223] Registration of the crio container factory successfully Jul 28 12:06:06.061168 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.061115 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 28 12:06:06.061168 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.061138 2575 factory.go:103] Registering Raw factory Jul 28 12:06:06.061168 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.061154 2575 manager.go:1196] Started watching for new ooms in manager Jul 28 12:06:06.061507 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.061492 2575 manager.go:349] Starting recovery of all containers Jul 28 12:06:06.067814 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.067791 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 28 12:06:06.067897 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.067874 2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-158.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 28 12:06:06.071677 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.071660 2575 manager.go:354] Recovery completed Jul 28 12:06:06.076134 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.076122 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 12:06:06.079432 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.079414 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasSufficientMemory" Jul 28 12:06:06.079517 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.079444 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 12:06:06.079517 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.079456 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasSufficientPID" Jul 28 12:06:06.080049 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.080032 2575 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 28 12:06:06.080049 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.080044 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 12:06:06.080049 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.080049 2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 28 12:06:06.080195 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.080067 2575 state_mem.go:36] "Initialized new in-memory state store" Jul 28 12:06:06.080801 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.080787 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasSufficientMemory" Jul 28 12:06:06.080861 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.080816 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 12:06:06.080861 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.080830 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasSufficientPID" Jul 28 12:06:06.082004 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.081993 2575 policy_none.go:49] "None policy: Start" Jul 28 12:06:06.082046 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.082009 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 28 12:06:06.082046 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.082019 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 28 12:06:06.083033 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.083023 2575 policy_none.go:47] "Start" Jul 28 12:06:06.117838 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.117816 2575 manager.go:337] "Starting Device Plugin manager" Jul 28 12:06:06.117929 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.117854 2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 28 12:06:06.117929 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.117867 2575 server.go:87] "Starting device plugin registration server" Jul 28 12:06:06.118143 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.118130 2575 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 28 12:06:06.118199 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.118145 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 28 12:06:06.118269 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.118245 2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 28 12:06:06.118367 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.118353 2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 28 12:06:06.118367 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.118364 2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 28 12:06:06.118778 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.118756 2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 28 12:06:06.118874 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.118801 2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-158.ec2.internal\" not found" Jul 28 12:06:06.201973 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.201934 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 28 12:06:06.203496 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.203479 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 28 12:06:06.203564 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.203525 2575 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 28 12:06:06.203564 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.203552 2575 kubelet.go:2448] "Starting kubelet main sync loop" Jul 28 12:06:06.203664 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.203589 2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 28 12:06:06.206190 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.206165 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 12:06:06.219011 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.218986 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 12:06:06.219957 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.219912 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasSufficientMemory" Jul 28 12:06:06.219957 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.219949 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 12:06:06.220057 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.219960 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasSufficientPID" Jul 28 12:06:06.220057 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.219984 2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.228520 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.228504 2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.228573 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.228528 2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-158.ec2.internal\": node \"ip-10-0-129-158.ec2.internal\" not found" Jul 28 12:06:06.246129 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.246108 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-158.ec2.internal\" not found" Jul 28 12:06:06.303924 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.303895 2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-158.ec2.internal"] Jul 28 12:06:06.304025 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.303965 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 12:06:06.305658 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.305642 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasSufficientMemory" Jul 28 12:06:06.305718 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.305673 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 12:06:06.305718 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.305688 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasSufficientPID" Jul 28 12:06:06.307442 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.307429 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 12:06:06.307583 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.307569 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.307627 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.307600 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 12:06:06.308181 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.308155 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasSufficientMemory" Jul 28 12:06:06.308273 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.308186 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 12:06:06.308273 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.308201 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasSufficientPID" Jul 28 12:06:06.308273 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.308206 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasSufficientMemory" Jul 28 12:06:06.308273 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.308223 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 12:06:06.308273 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.308232 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasSufficientPID" Jul 28 12:06:06.310049 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.310031 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.310105 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.310070 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 12:06:06.310822 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.310796 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasSufficientMemory" Jul 28 12:06:06.310893 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.310837 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 12:06:06.310893 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.310852 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeHasSufficientPID" Jul 28 12:06:06.335257 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.335230 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-158.ec2.internal\" not found" node="ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.340262 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.340243 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-158.ec2.internal\" not found" node="ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.346223 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.346208 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-158.ec2.internal\" not found" Jul 28 12:06:06.362471 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.362445 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b3d168e0b7bc31f63caad961d7a38d8f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal\" (UID: \"b3d168e0b7bc31f63caad961d7a38d8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.362548 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.362471 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3d168e0b7bc31f63caad961d7a38d8f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal\" (UID: \"b3d168e0b7bc31f63caad961d7a38d8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.362548 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.362490 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d1c563166026be471d3364cd537345ce-config\") pod \"kube-apiserver-proxy-ip-10-0-129-158.ec2.internal\" (UID: \"d1c563166026be471d3364cd537345ce\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.446751 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.446711 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-158.ec2.internal\" not found" Jul 28 12:06:06.463371 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.463345 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b3d168e0b7bc31f63caad961d7a38d8f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal\" (UID: \"b3d168e0b7bc31f63caad961d7a38d8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.463477 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.463380 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3d168e0b7bc31f63caad961d7a38d8f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal\" (UID: \"b3d168e0b7bc31f63caad961d7a38d8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.463477 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.463422 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d1c563166026be471d3364cd537345ce-config\") pod \"kube-apiserver-proxy-ip-10-0-129-158.ec2.internal\" (UID: \"d1c563166026be471d3364cd537345ce\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.463477 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.463461 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b3d168e0b7bc31f63caad961d7a38d8f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal\" (UID: \"b3d168e0b7bc31f63caad961d7a38d8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.463477 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.463470 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d1c563166026be471d3364cd537345ce-config\") pod \"kube-apiserver-proxy-ip-10-0-129-158.ec2.internal\" (UID: \"d1c563166026be471d3364cd537345ce\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.463636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.463467 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b3d168e0b7bc31f63caad961d7a38d8f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal\" (UID: \"b3d168e0b7bc31f63caad961d7a38d8f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.547747 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.547668 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-158.ec2.internal\" not found" Jul 28 12:06:06.638018 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.637827 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.644051 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.644026 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-158.ec2.internal" Jul 28 12:06:06.648350 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.648328 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-158.ec2.internal\" not found" Jul 28 12:06:06.750705 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.750666 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-158.ec2.internal\" not found" Jul 28 12:06:06.851811 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.851737 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-158.ec2.internal\" not found" Jul 28 12:06:06.951917 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:06.951861 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-158.ec2.internal\" not found" Jul 28 12:06:06.990692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.990659 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 28 12:06:06.991169 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:06.990827 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 12:06:07.052640 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:07.052607 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-158.ec2.internal\" not found" Jul 28 12:06:07.065425 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:07.065368 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 12:01:06 +0000 UTC" deadline="2027-12-25 19:32:17.88512698 +0000 UTC" Jul 28 12:06:07.065425 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:07.065417 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12367h26m10.819721311s" Jul 28 12:06:07.152837 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:07.152813 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-158.ec2.internal\" not found" Jul 28 12:06:07.153224 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:07.153196 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c563166026be471d3364cd537345ce.slice/crio-c6c6c4db5780392f081a82f976d0c751f8456c405201278fd23f982c5d478457 WatchSource:0}: Error finding container c6c6c4db5780392f081a82f976d0c751f8456c405201278fd23f982c5d478457: Status 404 returned error can't find the container with id c6c6c4db5780392f081a82f976d0c751f8456c405201278fd23f982c5d478457 Jul 28 12:06:07.153467 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:07.153447 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3d168e0b7bc31f63caad961d7a38d8f.slice/crio-1d7766e830730268046b799ab01a0b16141fd6f048dd36b235fe7f9b38facebf WatchSource:0}: Error finding container 1d7766e830730268046b799ab01a0b16141fd6f048dd36b235fe7f9b38facebf: Status 404 returned error can't find the container with id 1d7766e830730268046b799ab01a0b16141fd6f048dd36b235fe7f9b38facebf Jul 28 12:06:07.159810 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:07.159788 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 12:06:07.176537 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:07.176510 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 12:06:07.207222 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:07.207173 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-158.ec2.internal" event={"ID":"d1c563166026be471d3364cd537345ce","Type":"ContainerStarted","Data":"c6c6c4db5780392f081a82f976d0c751f8456c405201278fd23f982c5d478457"} Jul 28 12:06:07.208156 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:07.208134 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal" event={"ID":"b3d168e0b7bc31f63caad961d7a38d8f","Type":"ContainerStarted","Data":"1d7766e830730268046b799ab01a0b16141fd6f048dd36b235fe7f9b38facebf"} Jul 28 12:06:07.244117 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:07.244089 2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 12:06:07.253582 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:07.253551 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-158.ec2.internal\" not found" Jul 28 12:06:07.360196 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:07.360154 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-158.ec2.internal\" not found" Jul 28 12:06:07.463292 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:07.463254 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-158.ec2.internal\" not found" Jul 28 12:06:07.509758 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:07.509728 2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 12:06:07.560284 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:07.560244 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal" Jul 28 12:06:07.579884 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:07.579850 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 12:06:07.580967 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:07.580948 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-158.ec2.internal" Jul 28 12:06:07.592712 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:07.592684 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 12:06:08.033845 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.033810 2575 apiserver.go:52] "Watching apiserver" Jul 28 12:06:08.043753 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.043729 2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 28 12:06:08.045105 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.045077 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-fd7d6","openshift-ovn-kubernetes/ovnkube-node-v95xv","kube-system/konnectivity-agent-tzfpr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl","openshift-cluster-node-tuning-operator/tuned-cml7q","openshift-dns/node-resolver-n97lp","openshift-multus/multus-wnt8s","openshift-multus/network-metrics-daemon-5f72v","openshift-network-operator/iptables-alerter-276l8","kube-system/global-pull-secret-syncer-kkbtd","kube-system/kube-apiserver-proxy-ip-10-0-129-158.ec2.internal","openshift-image-registry/node-ca-d4d6q","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal","openshift-multus/multus-additional-cni-plugins-q2dw8"] Jul 28 12:06:08.046869 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.046848 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.048360 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.048336 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:08.048473 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.048416 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:08.049691 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.049671 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tzfpr" Jul 28 12:06:08.051017 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.050840 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 28 12:06:08.051017 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.050894 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 28 12:06:08.051224 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.051204 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.051369 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.051351 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 28 12:06:08.051457 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.051383 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 28 12:06:08.051457 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.051397 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mgvkm\"" Jul 28 12:06:08.052709 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.052691 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.053564 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.053361 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 28 12:06:08.053564 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.053371 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-n2xmw\"" Jul 28 12:06:08.053711 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.053662 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 28 12:06:08.053908 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.053890 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 28 12:06:08.054072 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.054054 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n97lp" Jul 28 12:06:08.054862 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.054721 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 28 12:06:08.054862 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.054733 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 28 12:06:08.054862 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.054772 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qz5td\"" Jul 28 12:06:08.055764 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.055329 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 28 12:06:08.055764 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.055570 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 28 12:06:08.055764 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.055642 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-msj8h\"" Jul 28 12:06:08.056697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.056678 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-276l8" Jul 28 12:06:08.058096 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.058075 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:08.058211 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.058171 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:08.058458 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.058435 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 28 12:06:08.058552 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.058523 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 28 12:06:08.058922 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.058764 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-crf9p\"" Jul 28 12:06:08.059943 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.059909 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 28 12:06:08.059943 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.059928 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 28 12:06:08.060116 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.059943 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 28 12:06:08.060181 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.060117 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-m9pnd\"" Jul 28 12:06:08.060181 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.060144 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 28 12:06:08.061361 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.061319 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.065756 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.065735 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:08.065872 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.065845 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:08.067067 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.067048 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 28 12:06:08.067726 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.067343 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 28 12:06:08.067726 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.067355 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 28 12:06:08.067726 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.067498 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 28 12:06:08.067726 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.067501 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 28 12:06:08.067726 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.067509 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 28 12:06:08.068267 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.068243 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xd5b5\"" Jul 28 12:06:08.068406 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.068387 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.069860 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.069840 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d4d6q" Jul 28 12:06:08.071241 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.071223 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 28 12:06:08.071528 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.071514 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gw884\"" Jul 28 12:06:08.071778 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.071760 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 28 12:06:08.072261 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.072242 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 28 12:06:08.072589 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.072567 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-69f9v\"" Jul 28 12:06:08.072708 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.072567 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 28 12:06:08.072881 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.072863 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 28 12:06:08.079699 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.079680 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ee66aa6f-353c-464a-9a45-a009910fc6e3-env-overrides\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.079800 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.079706 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpq47\" (UniqueName: \"kubernetes.io/projected/ee66aa6f-353c-464a-9a45-a009910fc6e3-kube-api-access-vpq47\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.079800 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.079722 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-registration-dir\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.079800 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.079744 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-etc-selinux\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.079800 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.079770 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-sysconfig\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.080022 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.079810 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-run\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.080022 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.079860 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-tmp\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.080022 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.079887 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-cnibin\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.080022 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.079915 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-run-k8s-cni-cncf-io\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.080022 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.079964 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr87b\" (UniqueName: \"kubernetes.io/projected/20658b18-e0e0-4a39-a49a-7b386a1600cb-kube-api-access-qr87b\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.080022 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.079998 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-modprobe-d\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.080331 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080024 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/20658b18-e0e0-4a39-a49a-7b386a1600cb-multus-daemon-config\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.080331 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080047 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.080331 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080064 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-kubernetes\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.080331 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080105 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-sys\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.080331 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080140 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1c56eeda-c493-4055-87eb-1d2fea4a014d-hosts-file\") pod \"node-resolver-n97lp\" (UID: \"1c56eeda-c493-4055-87eb-1d2fea4a014d\") " pod="openshift-dns/node-resolver-n97lp" Jul 28 12:06:08.080331 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080169 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-etc-openvswitch\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.080331 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080196 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ee66aa6f-353c-464a-9a45-a009910fc6e3-ovnkube-config\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.080331 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080221 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-run-multus-certs\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.080331 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080248 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-device-dir\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.080331 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080272 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-sysctl-conf\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.080331 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080296 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fe470f0-c27e-418f-af86-7fe9399cd0d4-host-slash\") pod \"iptables-alerter-276l8\" (UID: \"5fe470f0-c27e-418f-af86-7fe9399cd0d4\") " pod="openshift-network-operator/iptables-alerter-276l8" Jul 28 12:06:08.080821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080339 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6qbq\" (UniqueName: \"kubernetes.io/projected/5fe470f0-c27e-418f-af86-7fe9399cd0d4-kube-api-access-l6qbq\") pod \"iptables-alerter-276l8\" (UID: \"5fe470f0-c27e-418f-af86-7fe9399cd0d4\") " pod="openshift-network-operator/iptables-alerter-276l8" Jul 28 12:06:08.080821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080367 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-slash\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.080821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080392 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-run-netns\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.080821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080417 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-log-socket\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.080821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080441 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ee66aa6f-353c-464a-9a45-a009910fc6e3-ovnkube-script-lib\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.080821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080476 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-system-cni-dir\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.080821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080501 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs\") pod \"network-metrics-daemon-5f72v\" (UID: \"92daaf51-12ba-4eab-87ac-2087d60aac39\") " pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:08.080821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080527 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj2xr\" (UniqueName: \"kubernetes.io/projected/92daaf51-12ba-4eab-87ac-2087d60aac39-kube-api-access-vj2xr\") pod \"network-metrics-daemon-5f72v\" (UID: \"92daaf51-12ba-4eab-87ac-2087d60aac39\") " pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:08.080821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080553 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-lib-modules\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.080821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080577 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-run-systemd\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.080821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080625 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-cni-netd\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.080821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080657 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/00535017-b0ac-444e-ae72-711bf4003f21-agent-certs\") pod \"konnectivity-agent-tzfpr\" (UID: \"00535017-b0ac-444e-ae72-711bf4003f21\") " pod="kube-system/konnectivity-agent-tzfpr" Jul 28 12:06:08.080821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080679 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c56eeda-c493-4055-87eb-1d2fea4a014d-tmp-dir\") pod \"node-resolver-n97lp\" (UID: \"1c56eeda-c493-4055-87eb-1d2fea4a014d\") " pod="openshift-dns/node-resolver-n97lp" Jul 28 12:06:08.080821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080714 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee66aa6f-353c-464a-9a45-a009910fc6e3-ovn-node-metrics-cert\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.080821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080764 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-multus-conf-dir\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.080821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080810 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-socket-dir\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.081524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080838 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt2tw\" (UniqueName: \"kubernetes.io/projected/1c56eeda-c493-4055-87eb-1d2fea4a014d-kube-api-access-nt2tw\") pod \"node-resolver-n97lp\" (UID: \"1c56eeda-c493-4055-87eb-1d2fea4a014d\") " pod="openshift-dns/node-resolver-n97lp" Jul 28 12:06:08.081524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080873 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-systemd-units\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.081524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080900 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.081524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080936 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/00535017-b0ac-444e-ae72-711bf4003f21-konnectivity-ca\") pod \"konnectivity-agent-tzfpr\" (UID: \"00535017-b0ac-444e-ae72-711bf4003f21\") " pod="kube-system/konnectivity-agent-tzfpr" Jul 28 12:06:08.081524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080961 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-hostroot\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.081524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.080987 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftqtq\" (UniqueName: \"kubernetes.io/projected/4c4d1418-6b32-4534-b79c-5452b0d958d9-kube-api-access-ftqtq\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.081524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081011 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czjbs\" (UniqueName: \"kubernetes.io/projected/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-kube-api-access-czjbs\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.081524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081036 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-run-ovn-kubernetes\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.081524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081063 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-var-lib-cni-bin\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.081524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081095 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-var-lib-openvswitch\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.081524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081144 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-multus-cni-dir\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.081524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081169 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-os-release\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.081524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081185 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20658b18-e0e0-4a39-a49a-7b386a1600cb-cni-binary-copy\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.081524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081206 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-run-netns\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.081524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081226 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-var-lib-cni-multus\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.081524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081262 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-kubelet\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.082239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081292 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-node-log\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.082239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081340 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-multus-socket-dir-parent\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.082239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081365 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-etc-kubernetes\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.082239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081387 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-sysctl-d\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.082239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081410 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-tuned\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.082239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081435 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5fe470f0-c27e-418f-af86-7fe9399cd0d4-iptables-alerter-script\") pod \"iptables-alerter-276l8\" (UID: \"5fe470f0-c27e-418f-af86-7fe9399cd0d4\") " pod="openshift-network-operator/iptables-alerter-276l8" Jul 28 12:06:08.082239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081458 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-var-lib-kubelet\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.082239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081493 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-systemd\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.082239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081529 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-run-openvswitch\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.082239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081554 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-run-ovn\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.082239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081583 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-var-lib-kubelet\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.082239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081632 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-sys-fs\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.082239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081670 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-cni-bin\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.082239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081694 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-host\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.082239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081721 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nt9n\" (UniqueName: \"kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n\") pod \"network-check-target-fd7d6\" (UID: \"bd0bd704-8198-4017-a6d4-f8222009e54e\") " pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:08.082239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.081769 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 12:06:08.105465 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.105437 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bjx2c" Jul 28 12:06:08.117630 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.117607 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bjx2c" Jul 28 12:06:08.161013 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.160982 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 28 12:06:08.182375 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182344 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-lib-modules\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.182551 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182383 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-run-systemd\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.182551 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182409 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-cni-netd\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.182551 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182428 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/00535017-b0ac-444e-ae72-711bf4003f21-agent-certs\") pod \"konnectivity-agent-tzfpr\" (UID: \"00535017-b0ac-444e-ae72-711bf4003f21\") " pod="kube-system/konnectivity-agent-tzfpr" Jul 28 12:06:08.182551 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182448 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c56eeda-c493-4055-87eb-1d2fea4a014d-tmp-dir\") pod \"node-resolver-n97lp\" (UID: \"1c56eeda-c493-4055-87eb-1d2fea4a014d\") " pod="openshift-dns/node-resolver-n97lp" Jul 28 12:06:08.182551 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182471 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee66aa6f-353c-464a-9a45-a009910fc6e3-ovn-node-metrics-cert\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.182551 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182498 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-run-systemd\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.182551 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182517 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-cni-netd\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.182551 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182531 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-multus-conf-dir\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.182551 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182547 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-lib-modules\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182575 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-multus-conf-dir\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182572 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-socket-dir\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182620 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt2tw\" (UniqueName: \"kubernetes.io/projected/1c56eeda-c493-4055-87eb-1d2fea4a014d-kube-api-access-nt2tw\") pod \"node-resolver-n97lp\" (UID: \"1c56eeda-c493-4055-87eb-1d2fea4a014d\") " pod="openshift-dns/node-resolver-n97lp" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182647 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-kubelet-config\") pod \"global-pull-secret-syncer-kkbtd\" (UID: \"0f3bc13c-e0e8-44a6-9885-5bcd094112ac\") " pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182671 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-systemd-units\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182699 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-socket-dir\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182698 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182732 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/00535017-b0ac-444e-ae72-711bf4003f21-konnectivity-ca\") pod \"konnectivity-agent-tzfpr\" (UID: \"00535017-b0ac-444e-ae72-711bf4003f21\") " pod="kube-system/konnectivity-agent-tzfpr" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182739 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182756 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182774 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-hostroot\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182791 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftqtq\" (UniqueName: \"kubernetes.io/projected/4c4d1418-6b32-4534-b79c-5452b0d958d9-kube-api-access-ftqtq\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182799 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-systemd-units\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182808 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czjbs\" (UniqueName: \"kubernetes.io/projected/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-kube-api-access-czjbs\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182838 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz6h8\" (UniqueName: \"kubernetes.io/projected/adc764ba-183f-42ff-b0b9-475dcda06640-kube-api-access-gz6h8\") pod \"node-ca-d4d6q\" (UID: \"adc764ba-183f-42ff-b0b9-475dcda06640\") " pod="openshift-image-registry/node-ca-d4d6q" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182860 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1c56eeda-c493-4055-87eb-1d2fea4a014d-tmp-dir\") pod \"node-resolver-n97lp\" (UID: \"1c56eeda-c493-4055-87eb-1d2fea4a014d\") " pod="openshift-dns/node-resolver-n97lp" Jul 28 12:06:08.183010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182901 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-run-ovn-kubernetes\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182922 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-hostroot\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182870 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-run-ovn-kubernetes\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182882 2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.182978 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-var-lib-cni-bin\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183006 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-os-release\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183032 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-var-lib-openvswitch\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183055 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-multus-cni-dir\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183076 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-os-release\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183099 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20658b18-e0e0-4a39-a49a-7b386a1600cb-cni-binary-copy\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183095 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-var-lib-openvswitch\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183123 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-run-netns\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183145 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-var-lib-cni-multus\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183169 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-cnibin\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183173 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-os-release\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183194 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-kubelet\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183220 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-node-log\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183228 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-var-lib-cni-multus\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.183906 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183233 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-multus-cni-dir\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183244 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-multus-socket-dir-parent\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183286 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-run-netns\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183284 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-etc-kubernetes\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183312 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-kubelet\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183321 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-etc-kubernetes\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183328 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-sysctl-d\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183329 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/00535017-b0ac-444e-ae72-711bf4003f21-konnectivity-ca\") pod \"konnectivity-agent-tzfpr\" (UID: \"00535017-b0ac-444e-ae72-711bf4003f21\") " pod="kube-system/konnectivity-agent-tzfpr" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183354 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-tuned\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183391 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-node-log\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183404 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5fe470f0-c27e-418f-af86-7fe9399cd0d4-iptables-alerter-script\") pod \"iptables-alerter-276l8\" (UID: \"5fe470f0-c27e-418f-af86-7fe9399cd0d4\") " pod="openshift-network-operator/iptables-alerter-276l8" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183441 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret\") pod \"global-pull-secret-syncer-kkbtd\" (UID: \"0f3bc13c-e0e8-44a6-9885-5bcd094112ac\") " pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183493 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-var-lib-kubelet\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183518 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-systemd\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183537 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-sysctl-d\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183548 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-dbus\") pod \"global-pull-secret-syncer-kkbtd\" (UID: \"0f3bc13c-e0e8-44a6-9885-5bcd094112ac\") " pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183446 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-multus-socket-dir-parent\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.184787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183576 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-cni-binary-copy\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183601 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-systemd\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183602 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183633 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-var-lib-kubelet\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183637 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-run-openvswitch\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183665 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-run-ovn\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183677 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-run-openvswitch\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183686 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-var-lib-kubelet\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183710 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183736 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7m8r\" (UniqueName: \"kubernetes.io/projected/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-kube-api-access-l7m8r\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183747 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20658b18-e0e0-4a39-a49a-7b386a1600cb-cni-binary-copy\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183761 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-sys-fs\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183768 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-var-lib-kubelet\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183789 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-cni-bin\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183811 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-host\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183835 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nt9n\" (UniqueName: \"kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n\") pod \"network-check-target-fd7d6\" (UID: \"bd0bd704-8198-4017-a6d4-f8222009e54e\") " pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183837 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-run-ovn\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.185636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183838 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-sys-fs\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183866 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-cni-bin\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183879 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-host\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183912 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/adc764ba-183f-42ff-b0b9-475dcda06640-serviceca\") pod \"node-ca-d4d6q\" (UID: \"adc764ba-183f-42ff-b0b9-475dcda06640\") " pod="openshift-image-registry/node-ca-d4d6q" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183962 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ee66aa6f-353c-464a-9a45-a009910fc6e3-env-overrides\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183994 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpq47\" (UniqueName: \"kubernetes.io/projected/ee66aa6f-353c-464a-9a45-a009910fc6e3-kube-api-access-vpq47\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.183996 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5fe470f0-c27e-418f-af86-7fe9399cd0d4-iptables-alerter-script\") pod \"iptables-alerter-276l8\" (UID: \"5fe470f0-c27e-418f-af86-7fe9399cd0d4\") " pod="openshift-network-operator/iptables-alerter-276l8" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.184020 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-registration-dir\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.184045 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-etc-selinux\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.184070 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-sysconfig\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.184093 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-registration-dir\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.184095 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-run\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.184132 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-tmp\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.184155 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-cnibin\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.184899 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-run-k8s-cni-cncf-io\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.184958 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr87b\" (UniqueName: \"kubernetes.io/projected/20658b18-e0e0-4a39-a49a-7b386a1600cb-kube-api-access-qr87b\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.185002 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-modprobe-d\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.186521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.185211 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ee66aa6f-353c-464a-9a45-a009910fc6e3-env-overrides\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.185439 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-etc-selinux\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.185579 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-modprobe-d\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.184135 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-run\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186048 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-cnibin\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186128 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-sysconfig\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186248 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-var-lib-cni-bin\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186348 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-run-k8s-cni-cncf-io\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.185032 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/adc764ba-183f-42ff-b0b9-475dcda06640-host\") pod \"node-ca-d4d6q\" (UID: \"adc764ba-183f-42ff-b0b9-475dcda06640\") " pod="openshift-image-registry/node-ca-d4d6q" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186495 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/20658b18-e0e0-4a39-a49a-7b386a1600cb-multus-daemon-config\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186521 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186560 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-kubernetes\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186588 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-sys\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186616 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1c56eeda-c493-4055-87eb-1d2fea4a014d-hosts-file\") pod \"node-resolver-n97lp\" (UID: \"1c56eeda-c493-4055-87eb-1d2fea4a014d\") " pod="openshift-dns/node-resolver-n97lp" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186642 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-etc-openvswitch\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186672 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ee66aa6f-353c-464a-9a45-a009910fc6e3-ovnkube-config\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186703 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-run-multus-certs\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186733 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-device-dir\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.187417 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186817 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-sysctl-conf\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186845 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fe470f0-c27e-418f-af86-7fe9399cd0d4-host-slash\") pod \"iptables-alerter-276l8\" (UID: \"5fe470f0-c27e-418f-af86-7fe9399cd0d4\") " pod="openshift-network-operator/iptables-alerter-276l8" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186876 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6qbq\" (UniqueName: \"kubernetes.io/projected/5fe470f0-c27e-418f-af86-7fe9399cd0d4-kube-api-access-l6qbq\") pod \"iptables-alerter-276l8\" (UID: \"5fe470f0-c27e-418f-af86-7fe9399cd0d4\") " pod="openshift-network-operator/iptables-alerter-276l8" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186910 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-system-cni-dir\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186944 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-slash\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186973 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-run-netns\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.186999 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-log-socket\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.187028 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ee66aa6f-353c-464a-9a45-a009910fc6e3-ovnkube-script-lib\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.187058 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-system-cni-dir\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.187092 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs\") pod \"network-metrics-daemon-5f72v\" (UID: \"92daaf51-12ba-4eab-87ac-2087d60aac39\") " pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.187118 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj2xr\" (UniqueName: \"kubernetes.io/projected/92daaf51-12ba-4eab-87ac-2087d60aac39-kube-api-access-vj2xr\") pod \"network-metrics-daemon-5f72v\" (UID: \"92daaf51-12ba-4eab-87ac-2087d60aac39\") " pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.187592 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/00535017-b0ac-444e-ae72-711bf4003f21-agent-certs\") pod \"konnectivity-agent-tzfpr\" (UID: \"00535017-b0ac-444e-ae72-711bf4003f21\") " pod="kube-system/konnectivity-agent-tzfpr" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.187743 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-sysctl-conf\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.187815 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.187878 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-kubernetes\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.187896 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/20658b18-e0e0-4a39-a49a-7b386a1600cb-multus-daemon-config\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.187930 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-sys\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.188318 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.187968 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-etc-tuned\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.189088 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.187990 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-etc-openvswitch\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.189088 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.188058 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-log-socket\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.189088 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.188065 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4c4d1418-6b32-4534-b79c-5452b0d958d9-device-dir\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.189088 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.188149 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-slash\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.189088 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.188181 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fe470f0-c27e-418f-af86-7fe9399cd0d4-host-slash\") pod \"iptables-alerter-276l8\" (UID: \"5fe470f0-c27e-418f-af86-7fe9399cd0d4\") " pod="openshift-network-operator/iptables-alerter-276l8" Jul 28 12:06:08.189088 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.188191 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1c56eeda-c493-4055-87eb-1d2fea4a014d-hosts-file\") pod \"node-resolver-n97lp\" (UID: \"1c56eeda-c493-4055-87eb-1d2fea4a014d\") " pod="openshift-dns/node-resolver-n97lp" Jul 28 12:06:08.189088 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.188202 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-host-run-multus-certs\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.189088 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.188250 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ee66aa6f-353c-464a-9a45-a009910fc6e3-host-run-netns\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.189088 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.188288 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 12:06:08.189088 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.188391 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs podName:92daaf51-12ba-4eab-87ac-2087d60aac39 nodeName:}" failed. No retries permitted until 2026-07-28 12:06:08.688362598 +0000 UTC m=+3.013027882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs") pod "network-metrics-daemon-5f72v" (UID: "92daaf51-12ba-4eab-87ac-2087d60aac39") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 12:06:08.189088 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.188966 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ee66aa6f-353c-464a-9a45-a009910fc6e3-ovn-node-metrics-cert\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.189088 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.189043 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ee66aa6f-353c-464a-9a45-a009910fc6e3-ovnkube-config\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.189725 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.189104 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-tmp\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.189725 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.189115 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20658b18-e0e0-4a39-a49a-7b386a1600cb-system-cni-dir\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.190361 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.190333 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ee66aa6f-353c-464a-9a45-a009910fc6e3-ovnkube-script-lib\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.195107 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.195081 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt2tw\" (UniqueName: \"kubernetes.io/projected/1c56eeda-c493-4055-87eb-1d2fea4a014d-kube-api-access-nt2tw\") pod \"node-resolver-n97lp\" (UID: \"1c56eeda-c493-4055-87eb-1d2fea4a014d\") " pod="openshift-dns/node-resolver-n97lp" Jul 28 12:06:08.195487 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.195462 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftqtq\" (UniqueName: \"kubernetes.io/projected/4c4d1418-6b32-4534-b79c-5452b0d958d9-kube-api-access-ftqtq\") pod \"aws-ebs-csi-driver-node-9hhjl\" (UID: \"4c4d1418-6b32-4534-b79c-5452b0d958d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.199847 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.199828 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr87b\" (UniqueName: \"kubernetes.io/projected/20658b18-e0e0-4a39-a49a-7b386a1600cb-kube-api-access-qr87b\") pod \"multus-wnt8s\" (UID: \"20658b18-e0e0-4a39-a49a-7b386a1600cb\") " pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.200527 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.200507 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 12:06:08.200622 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.200532 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 12:06:08.200622 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.200544 2575 projected.go:196] Error preparing data for projected volume kube-api-access-6nt9n for pod openshift-network-diagnostics/network-check-target-fd7d6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 12:06:08.200737 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.200639 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n podName:bd0bd704-8198-4017-a6d4-f8222009e54e nodeName:}" failed. No retries permitted until 2026-07-28 12:06:08.700619666 +0000 UTC m=+3.025284947 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6nt9n" (UniqueName: "kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n") pod "network-check-target-fd7d6" (UID: "bd0bd704-8198-4017-a6d4-f8222009e54e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 12:06:08.203168 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.203143 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj2xr\" (UniqueName: \"kubernetes.io/projected/92daaf51-12ba-4eab-87ac-2087d60aac39-kube-api-access-vj2xr\") pod \"network-metrics-daemon-5f72v\" (UID: \"92daaf51-12ba-4eab-87ac-2087d60aac39\") " pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:08.203538 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.203513 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpq47\" (UniqueName: \"kubernetes.io/projected/ee66aa6f-353c-464a-9a45-a009910fc6e3-kube-api-access-vpq47\") pod \"ovnkube-node-v95xv\" (UID: \"ee66aa6f-353c-464a-9a45-a009910fc6e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.204881 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.204857 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6qbq\" (UniqueName: \"kubernetes.io/projected/5fe470f0-c27e-418f-af86-7fe9399cd0d4-kube-api-access-l6qbq\") pod \"iptables-alerter-276l8\" (UID: \"5fe470f0-c27e-418f-af86-7fe9399cd0d4\") " pod="openshift-network-operator/iptables-alerter-276l8" Jul 28 12:06:08.205602 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.205577 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czjbs\" (UniqueName: \"kubernetes.io/projected/bd19a5ef-beec-427f-8a40-5c0f076a8c1d-kube-api-access-czjbs\") pod \"tuned-cml7q\" (UID: \"bd19a5ef-beec-427f-8a40-5c0f076a8c1d\") " pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.287548 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287449 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-kubelet-config\") pod \"global-pull-secret-syncer-kkbtd\" (UID: \"0f3bc13c-e0e8-44a6-9885-5bcd094112ac\") " pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:08.287548 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287501 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.287548 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287529 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz6h8\" (UniqueName: \"kubernetes.io/projected/adc764ba-183f-42ff-b0b9-475dcda06640-kube-api-access-gz6h8\") pod \"node-ca-d4d6q\" (UID: \"adc764ba-183f-42ff-b0b9-475dcda06640\") " pod="openshift-image-registry/node-ca-d4d6q" Jul 28 12:06:08.287548 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287551 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-os-release\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.287880 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287575 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-cnibin\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.287880 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287600 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret\") pod \"global-pull-secret-syncer-kkbtd\" (UID: \"0f3bc13c-e0e8-44a6-9885-5bcd094112ac\") " pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:08.287880 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287594 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-kubelet-config\") pod \"global-pull-secret-syncer-kkbtd\" (UID: \"0f3bc13c-e0e8-44a6-9885-5bcd094112ac\") " pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:08.287880 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287617 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-dbus\") pod \"global-pull-secret-syncer-kkbtd\" (UID: \"0f3bc13c-e0e8-44a6-9885-5bcd094112ac\") " pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:08.287880 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287658 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-cni-binary-copy\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.287880 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287683 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-dbus\") pod \"global-pull-secret-syncer-kkbtd\" (UID: \"0f3bc13c-e0e8-44a6-9885-5bcd094112ac\") " pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:08.287880 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287685 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-cnibin\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.287880 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287685 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.287880 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287738 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.287880 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287766 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7m8r\" (UniqueName: \"kubernetes.io/projected/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-kube-api-access-l7m8r\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.287880 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287770 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-os-release\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.287880 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287810 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/adc764ba-183f-42ff-b0b9-475dcda06640-serviceca\") pod \"node-ca-d4d6q\" (UID: \"adc764ba-183f-42ff-b0b9-475dcda06640\") " pod="openshift-image-registry/node-ca-d4d6q" Jul 28 12:06:08.287880 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287848 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/adc764ba-183f-42ff-b0b9-475dcda06640-host\") pod \"node-ca-d4d6q\" (UID: \"adc764ba-183f-42ff-b0b9-475dcda06640\") " pod="openshift-image-registry/node-ca-d4d6q" Jul 28 12:06:08.287880 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.287861 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 12:06:08.287880 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287887 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-system-cni-dir\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.288577 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.287927 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret podName:0f3bc13c-e0e8-44a6-9885-5bcd094112ac nodeName:}" failed. No retries permitted until 2026-07-28 12:06:08.787909692 +0000 UTC m=+3.112574958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret") pod "global-pull-secret-syncer-kkbtd" (UID: "0f3bc13c-e0e8-44a6-9885-5bcd094112ac") : object "kube-system"/"original-pull-secret" not registered Jul 28 12:06:08.288577 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.287947 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-system-cni-dir\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.288577 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.288022 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/adc764ba-183f-42ff-b0b9-475dcda06640-host\") pod \"node-ca-d4d6q\" (UID: \"adc764ba-183f-42ff-b0b9-475dcda06640\") " pod="openshift-image-registry/node-ca-d4d6q" Jul 28 12:06:08.288577 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.288138 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.288577 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.288242 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.288577 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.288395 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/adc764ba-183f-42ff-b0b9-475dcda06640-serviceca\") pod \"node-ca-d4d6q\" (UID: \"adc764ba-183f-42ff-b0b9-475dcda06640\") " pod="openshift-image-registry/node-ca-d4d6q" Jul 28 12:06:08.288823 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.288640 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-cni-binary-copy\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.288823 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.288627 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.306238 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.306212 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7m8r\" (UniqueName: \"kubernetes.io/projected/7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425-kube-api-access-l7m8r\") pod \"multus-additional-cni-plugins-q2dw8\" (UID: \"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425\") " pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.311746 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.311720 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz6h8\" (UniqueName: \"kubernetes.io/projected/adc764ba-183f-42ff-b0b9-475dcda06640-kube-api-access-gz6h8\") pod \"node-ca-d4d6q\" (UID: \"adc764ba-183f-42ff-b0b9-475dcda06640\") " pod="openshift-image-registry/node-ca-d4d6q" Jul 28 12:06:08.360073 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.360036 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wnt8s" Jul 28 12:06:08.371120 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.371091 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tzfpr" Jul 28 12:06:08.380056 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.380032 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" Jul 28 12:06:08.385269 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.385247 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cml7q" Jul 28 12:06:08.391598 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.391576 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n97lp" Jul 28 12:06:08.400430 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.400409 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-276l8" Jul 28 12:06:08.406300 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.406253 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:08.413486 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.413465 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q2dw8" Jul 28 12:06:08.419439 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.419418 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d4d6q" Jul 28 12:06:08.440085 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.440063 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 12:06:08.691055 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.691022 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs\") pod \"network-metrics-daemon-5f72v\" (UID: \"92daaf51-12ba-4eab-87ac-2087d60aac39\") " pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:08.691250 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.691145 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 12:06:08.691250 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.691217 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs podName:92daaf51-12ba-4eab-87ac-2087d60aac39 nodeName:}" failed. No retries permitted until 2026-07-28 12:06:09.691195669 +0000 UTC m=+4.015860951 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs") pod "network-metrics-daemon-5f72v" (UID: "92daaf51-12ba-4eab-87ac-2087d60aac39") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 12:06:08.791627 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.791596 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret\") pod \"global-pull-secret-syncer-kkbtd\" (UID: \"0f3bc13c-e0e8-44a6-9885-5bcd094112ac\") " pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:08.791779 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:08.791639 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nt9n\" (UniqueName: \"kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n\") pod \"network-check-target-fd7d6\" (UID: \"bd0bd704-8198-4017-a6d4-f8222009e54e\") " pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:08.791779 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.791766 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 12:06:08.791894 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.791844 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret podName:0f3bc13c-e0e8-44a6-9885-5bcd094112ac nodeName:}" failed. No retries permitted until 2026-07-28 12:06:09.791822407 +0000 UTC m=+4.116487665 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret") pod "global-pull-secret-syncer-kkbtd" (UID: "0f3bc13c-e0e8-44a6-9885-5bcd094112ac") : object "kube-system"/"original-pull-secret" not registered Jul 28 12:06:08.791894 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.791770 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 12:06:08.791894 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.791891 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 12:06:08.792034 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.791906 2575 projected.go:196] Error preparing data for projected volume kube-api-access-6nt9n for pod openshift-network-diagnostics/network-check-target-fd7d6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 12:06:08.792637 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:08.792605 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n podName:bd0bd704-8198-4017-a6d4-f8222009e54e nodeName:}" failed. No retries permitted until 2026-07-28 12:06:09.792588848 +0000 UTC m=+4.117254122 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6nt9n" (UniqueName: "kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n") pod "network-check-target-fd7d6" (UID: "bd0bd704-8198-4017-a6d4-f8222009e54e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 12:06:09.101075 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:09.101044 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd19a5ef_beec_427f_8a40_5c0f076a8c1d.slice/crio-030cd9dd56f64dedc9e8d9f9136ba3086f5590f7ad093b77c8b7e392e0996706 WatchSource:0}: Error finding container 030cd9dd56f64dedc9e8d9f9136ba3086f5590f7ad093b77c8b7e392e0996706: Status 404 returned error can't find the container with id 030cd9dd56f64dedc9e8d9f9136ba3086f5590f7ad093b77c8b7e392e0996706 Jul 28 12:06:09.103810 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:09.103768 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20658b18_e0e0_4a39_a49a_7b386a1600cb.slice/crio-a5e2f30d139f6529f641f7108c2bd813ca74dc4e25e33dc444c48cd43963e082 WatchSource:0}: Error finding container a5e2f30d139f6529f641f7108c2bd813ca74dc4e25e33dc444c48cd43963e082: Status 404 returned error can't find the container with id a5e2f30d139f6529f641f7108c2bd813ca74dc4e25e33dc444c48cd43963e082 Jul 28 12:06:09.104349 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:09.104283 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c4d1418_6b32_4534_b79c_5452b0d958d9.slice/crio-2202f617924bf713e95e94890b0b61fbfe17da787339d65d47923ce688147efe WatchSource:0}: Error finding container 2202f617924bf713e95e94890b0b61fbfe17da787339d65d47923ce688147efe: Status 404 returned error can't find the container with id 2202f617924bf713e95e94890b0b61fbfe17da787339d65d47923ce688147efe Jul 28 12:06:09.104655 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:09.104625 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee66aa6f_353c_464a_9a45_a009910fc6e3.slice/crio-37f14039ad05d6864e53992e10abb94dc54477d42aee363deecd1adbf5caba9a WatchSource:0}: Error finding container 37f14039ad05d6864e53992e10abb94dc54477d42aee363deecd1adbf5caba9a: Status 404 returned error can't find the container with id 37f14039ad05d6864e53992e10abb94dc54477d42aee363deecd1adbf5caba9a Jul 28 12:06:09.107471 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:09.107442 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fe470f0_c27e_418f_af86_7fe9399cd0d4.slice/crio-8f4c8a8dd00720cd03270a6665857435772a48bd10188d4f897aa45f60408f25 WatchSource:0}: Error finding container 8f4c8a8dd00720cd03270a6665857435772a48bd10188d4f897aa45f60408f25: Status 404 returned error can't find the container with id 8f4c8a8dd00720cd03270a6665857435772a48bd10188d4f897aa45f60408f25 Jul 28 12:06:09.110238 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:09.110125 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00535017_b0ac_444e_ae72_711bf4003f21.slice/crio-8c723fa0601f2d9188970c0e76014e2c0699ecb1f158ec4070f116093c28944a WatchSource:0}: Error finding container 8c723fa0601f2d9188970c0e76014e2c0699ecb1f158ec4070f116093c28944a: Status 404 returned error can't find the container with id 8c723fa0601f2d9188970c0e76014e2c0699ecb1f158ec4070f116093c28944a Jul 28 12:06:09.111382 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:09.111352 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c56eeda_c493_4055_87eb_1d2fea4a014d.slice/crio-3c892c6284e7343f3d49b006dadd9b09aee17b3a64a04c2f52744830b1f4b2cc WatchSource:0}: Error finding container 3c892c6284e7343f3d49b006dadd9b09aee17b3a64a04c2f52744830b1f4b2cc: Status 404 returned error can't find the container with id 3c892c6284e7343f3d49b006dadd9b09aee17b3a64a04c2f52744830b1f4b2cc Jul 28 12:06:09.112509 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:09.112474 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadc764ba_183f_42ff_b0b9_475dcda06640.slice/crio-2146336084a4f50e1f6cd3e8117ee9906741cad2c38759e6e064d34e25b2a813 WatchSource:0}: Error finding container 2146336084a4f50e1f6cd3e8117ee9906741cad2c38759e6e064d34e25b2a813: Status 404 returned error can't find the container with id 2146336084a4f50e1f6cd3e8117ee9906741cad2c38759e6e064d34e25b2a813 Jul 28 12:06:09.114274 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:09.114217 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e8d0d2e_73f0_4971_bf1d_cd1fde0b3425.slice/crio-fc1242b079ffa2a09e7382511b2fa94321fc420f1ba2f0666a90bad83630febf WatchSource:0}: Error finding container fc1242b079ffa2a09e7382511b2fa94321fc420f1ba2f0666a90bad83630febf: Status 404 returned error can't find the container with id fc1242b079ffa2a09e7382511b2fa94321fc420f1ba2f0666a90bad83630febf Jul 28 12:06:09.119785 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:09.119732 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 12:01:08 +0000 UTC" deadline="2028-04-16 17:01:26.02987683 +0000 UTC" Jul 28 12:06:09.119852 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:09.119784 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15076h55m16.91009689s" Jul 28 12:06:09.213591 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:09.213553 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-158.ec2.internal" event={"ID":"d1c563166026be471d3364cd537345ce","Type":"ContainerStarted","Data":"81749ee370c16566cfdde0dd5874581e78bd38b461c12610549bbb160316b39f"} Jul 28 12:06:09.214730 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:09.214701 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tzfpr" event={"ID":"00535017-b0ac-444e-ae72-711bf4003f21","Type":"ContainerStarted","Data":"8c723fa0601f2d9188970c0e76014e2c0699ecb1f158ec4070f116093c28944a"} Jul 28 12:06:09.215838 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:09.215813 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cml7q" event={"ID":"bd19a5ef-beec-427f-8a40-5c0f076a8c1d","Type":"ContainerStarted","Data":"030cd9dd56f64dedc9e8d9f9136ba3086f5590f7ad093b77c8b7e392e0996706"} Jul 28 12:06:09.217181 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:09.217089 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2dw8" event={"ID":"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425","Type":"ContainerStarted","Data":"fc1242b079ffa2a09e7382511b2fa94321fc420f1ba2f0666a90bad83630febf"} Jul 28 12:06:09.218423 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:09.218388 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d4d6q" event={"ID":"adc764ba-183f-42ff-b0b9-475dcda06640","Type":"ContainerStarted","Data":"2146336084a4f50e1f6cd3e8117ee9906741cad2c38759e6e064d34e25b2a813"} Jul 28 12:06:09.219537 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:09.219512 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n97lp" event={"ID":"1c56eeda-c493-4055-87eb-1d2fea4a014d","Type":"ContainerStarted","Data":"3c892c6284e7343f3d49b006dadd9b09aee17b3a64a04c2f52744830b1f4b2cc"} Jul 28 12:06:09.220814 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:09.220766 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-276l8" event={"ID":"5fe470f0-c27e-418f-af86-7fe9399cd0d4","Type":"ContainerStarted","Data":"8f4c8a8dd00720cd03270a6665857435772a48bd10188d4f897aa45f60408f25"} Jul 28 12:06:09.222003 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:09.221982 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" event={"ID":"ee66aa6f-353c-464a-9a45-a009910fc6e3","Type":"ContainerStarted","Data":"37f14039ad05d6864e53992e10abb94dc54477d42aee363deecd1adbf5caba9a"} Jul 28 12:06:09.223125 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:09.223101 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" event={"ID":"4c4d1418-6b32-4534-b79c-5452b0d958d9","Type":"ContainerStarted","Data":"2202f617924bf713e95e94890b0b61fbfe17da787339d65d47923ce688147efe"} Jul 28 12:06:09.226462 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:09.226441 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wnt8s" event={"ID":"20658b18-e0e0-4a39-a49a-7b386a1600cb","Type":"ContainerStarted","Data":"a5e2f30d139f6529f641f7108c2bd813ca74dc4e25e33dc444c48cd43963e082"} Jul 28 12:06:09.233648 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:09.233603 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-158.ec2.internal" podStartSLOduration=2.233589566 podStartE2EDuration="2.233589566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:06:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:06:09.233063257 +0000 UTC m=+3.557728539" watchObservedRunningTime="2026-07-28 12:06:09.233589566 +0000 UTC m=+3.558254846" Jul 28 12:06:09.698542 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:09.698460 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs\") pod \"network-metrics-daemon-5f72v\" (UID: \"92daaf51-12ba-4eab-87ac-2087d60aac39\") " pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:09.698752 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:09.698688 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 12:06:09.698821 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:09.698761 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs podName:92daaf51-12ba-4eab-87ac-2087d60aac39 nodeName:}" failed. No retries permitted until 2026-07-28 12:06:11.698740979 +0000 UTC m=+6.023406237 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs") pod "network-metrics-daemon-5f72v" (UID: "92daaf51-12ba-4eab-87ac-2087d60aac39") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 12:06:09.799352 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:09.799293 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret\") pod \"global-pull-secret-syncer-kkbtd\" (UID: \"0f3bc13c-e0e8-44a6-9885-5bcd094112ac\") " pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:09.799537 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:09.799373 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nt9n\" (UniqueName: \"kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n\") pod \"network-check-target-fd7d6\" (UID: \"bd0bd704-8198-4017-a6d4-f8222009e54e\") " pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:09.799625 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:09.799561 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 12:06:09.799625 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:09.799582 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 12:06:09.799625 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:09.799597 2575 projected.go:196] Error preparing data for projected volume kube-api-access-6nt9n for pod openshift-network-diagnostics/network-check-target-fd7d6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 12:06:09.799791 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:09.799663 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n podName:bd0bd704-8198-4017-a6d4-f8222009e54e nodeName:}" failed. No retries permitted until 2026-07-28 12:06:11.799641741 +0000 UTC m=+6.124307012 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6nt9n" (UniqueName: "kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n") pod "network-check-target-fd7d6" (UID: "bd0bd704-8198-4017-a6d4-f8222009e54e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 12:06:09.800132 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:09.800108 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 12:06:09.800203 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:09.800169 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret podName:0f3bc13c-e0e8-44a6-9885-5bcd094112ac nodeName:}" failed. No retries permitted until 2026-07-28 12:06:11.80015466 +0000 UTC m=+6.124819922 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret") pod "global-pull-secret-syncer-kkbtd" (UID: "0f3bc13c-e0e8-44a6-9885-5bcd094112ac") : object "kube-system"/"original-pull-secret" not registered Jul 28 12:06:10.120581 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:10.120532 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 12:01:08 +0000 UTC" deadline="2028-03-10 20:38:53.818102539 +0000 UTC" Jul 28 12:06:10.120581 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:10.120575 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14192h32m43.697531989s" Jul 28 12:06:10.207478 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:10.207376 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:10.207697 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:10.207546 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:10.208036 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:10.208016 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:10.208145 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:10.208122 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:10.208266 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:10.208250 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:10.208379 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:10.208347 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:10.240075 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:10.240021 2575 generic.go:356] "Generic (PLEG): container finished" podID="b3d168e0b7bc31f63caad961d7a38d8f" containerID="f14b2ea9f89d4a84006cd33832e4a150c258e5a5fa28e71f22d160c779daddf4" exitCode=0 Jul 28 12:06:10.241084 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:10.241034 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal" event={"ID":"b3d168e0b7bc31f63caad961d7a38d8f","Type":"ContainerDied","Data":"f14b2ea9f89d4a84006cd33832e4a150c258e5a5fa28e71f22d160c779daddf4"} Jul 28 12:06:11.262770 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:11.262725 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal" event={"ID":"b3d168e0b7bc31f63caad961d7a38d8f","Type":"ContainerStarted","Data":"c14adfb43c3dd0865d474b430f958dbc9bc6f1596653d6941d2482fccd8a90dd"} Jul 28 12:06:11.715938 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:11.715893 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs\") pod \"network-metrics-daemon-5f72v\" (UID: \"92daaf51-12ba-4eab-87ac-2087d60aac39\") " pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:11.716187 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:11.716119 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 12:06:11.716272 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:11.716195 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs podName:92daaf51-12ba-4eab-87ac-2087d60aac39 nodeName:}" failed. No retries permitted until 2026-07-28 12:06:15.71617108 +0000 UTC m=+10.040836353 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs") pod "network-metrics-daemon-5f72v" (UID: "92daaf51-12ba-4eab-87ac-2087d60aac39") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 12:06:11.816798 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:11.816744 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret\") pod \"global-pull-secret-syncer-kkbtd\" (UID: \"0f3bc13c-e0e8-44a6-9885-5bcd094112ac\") " pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:11.816983 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:11.816816 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nt9n\" (UniqueName: \"kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n\") pod \"network-check-target-fd7d6\" (UID: \"bd0bd704-8198-4017-a6d4-f8222009e54e\") " pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:11.817069 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:11.817011 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 12:06:11.817069 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:11.817026 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 12:06:11.817069 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:11.817036 2575 projected.go:196] Error preparing data for projected volume kube-api-access-6nt9n for pod openshift-network-diagnostics/network-check-target-fd7d6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 12:06:11.817244 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:11.817101 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n podName:bd0bd704-8198-4017-a6d4-f8222009e54e nodeName:}" failed. No retries permitted until 2026-07-28 12:06:15.817078982 +0000 UTC m=+10.141744264 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6nt9n" (UniqueName: "kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n") pod "network-check-target-fd7d6" (UID: "bd0bd704-8198-4017-a6d4-f8222009e54e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 12:06:11.817590 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:11.817557 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 12:06:11.817680 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:11.817617 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret podName:0f3bc13c-e0e8-44a6-9885-5bcd094112ac nodeName:}" failed. No retries permitted until 2026-07-28 12:06:15.817599759 +0000 UTC m=+10.142265020 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret") pod "global-pull-secret-syncer-kkbtd" (UID: "0f3bc13c-e0e8-44a6-9885-5bcd094112ac") : object "kube-system"/"original-pull-secret" not registered Jul 28 12:06:12.206055 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:12.205320 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:12.206055 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:12.205495 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:12.206055 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:12.205918 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:12.206055 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:12.206034 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:12.206479 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:12.206172 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:12.206479 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:12.206259 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:14.205992 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:14.205960 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:14.206488 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:14.206082 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:14.206488 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:14.206121 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:14.206488 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:14.206207 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:14.206488 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:14.206247 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:14.206488 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:14.206322 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:15.754643 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:15.750753 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs\") pod \"network-metrics-daemon-5f72v\" (UID: \"92daaf51-12ba-4eab-87ac-2087d60aac39\") " pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:15.754643 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:15.750939 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 12:06:15.754643 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:15.751003 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs podName:92daaf51-12ba-4eab-87ac-2087d60aac39 nodeName:}" failed. No retries permitted until 2026-07-28 12:06:23.750984111 +0000 UTC m=+18.075649374 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs") pod "network-metrics-daemon-5f72v" (UID: "92daaf51-12ba-4eab-87ac-2087d60aac39") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 12:06:15.851579 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:15.851544 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret\") pod \"global-pull-secret-syncer-kkbtd\" (UID: \"0f3bc13c-e0e8-44a6-9885-5bcd094112ac\") " pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:15.851579 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:15.851592 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nt9n\" (UniqueName: \"kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n\") pod \"network-check-target-fd7d6\" (UID: \"bd0bd704-8198-4017-a6d4-f8222009e54e\") " pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:15.851843 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:15.851704 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 12:06:15.851843 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:15.851771 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret podName:0f3bc13c-e0e8-44a6-9885-5bcd094112ac nodeName:}" failed. No retries permitted until 2026-07-28 12:06:23.851753176 +0000 UTC m=+18.176418444 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret") pod "global-pull-secret-syncer-kkbtd" (UID: "0f3bc13c-e0e8-44a6-9885-5bcd094112ac") : object "kube-system"/"original-pull-secret" not registered Jul 28 12:06:15.851843 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:15.851708 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 12:06:15.851843 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:15.851817 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 12:06:15.851843 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:15.851830 2575 projected.go:196] Error preparing data for projected volume kube-api-access-6nt9n for pod openshift-network-diagnostics/network-check-target-fd7d6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 12:06:15.852215 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:15.851884 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n podName:bd0bd704-8198-4017-a6d4-f8222009e54e nodeName:}" failed. No retries permitted until 2026-07-28 12:06:23.851868383 +0000 UTC m=+18.176533665 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6nt9n" (UniqueName: "kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n") pod "network-check-target-fd7d6" (UID: "bd0bd704-8198-4017-a6d4-f8222009e54e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 12:06:16.206140 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:16.205419 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:16.206140 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:16.205532 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:16.206140 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:16.205899 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:16.206140 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:16.205991 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:16.206140 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:16.206047 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:16.206140 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:16.206113 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:18.205437 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:18.205398 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:18.205817 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:18.205398 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:18.205817 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:18.205548 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:18.205817 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:18.205612 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:18.205817 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:18.205634 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:18.205817 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:18.205713 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:20.204826 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:20.204784 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:20.205269 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:20.204784 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:20.205269 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:20.204936 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:20.205269 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:20.204933 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:20.205269 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:20.205031 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:20.205269 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:20.205153 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:22.204671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:22.204638 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:22.205134 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:22.204648 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:22.205134 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:22.204769 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:22.205134 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:22.204652 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:22.205134 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:22.204858 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:22.205134 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:22.204956 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:23.814009 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:23.813916 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs\") pod \"network-metrics-daemon-5f72v\" (UID: \"92daaf51-12ba-4eab-87ac-2087d60aac39\") " pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:23.814433 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:23.814080 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 12:06:23.814433 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:23.814151 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs podName:92daaf51-12ba-4eab-87ac-2087d60aac39 nodeName:}" failed. No retries permitted until 2026-07-28 12:06:39.814133252 +0000 UTC m=+34.138798531 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs") pod "network-metrics-daemon-5f72v" (UID: "92daaf51-12ba-4eab-87ac-2087d60aac39") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 12:06:23.915203 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:23.915169 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret\") pod \"global-pull-secret-syncer-kkbtd\" (UID: \"0f3bc13c-e0e8-44a6-9885-5bcd094112ac\") " pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:23.915406 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:23.915226 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nt9n\" (UniqueName: \"kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n\") pod \"network-check-target-fd7d6\" (UID: \"bd0bd704-8198-4017-a6d4-f8222009e54e\") " pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:23.915406 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:23.915378 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 12:06:23.915406 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:23.915403 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 12:06:23.915535 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:23.915418 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 12:06:23.915535 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:23.915431 2575 projected.go:196] Error preparing data for projected volume kube-api-access-6nt9n for pod openshift-network-diagnostics/network-check-target-fd7d6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 12:06:23.915535 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:23.915456 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret podName:0f3bc13c-e0e8-44a6-9885-5bcd094112ac nodeName:}" failed. No retries permitted until 2026-07-28 12:06:39.915433991 +0000 UTC m=+34.240099267 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret") pod "global-pull-secret-syncer-kkbtd" (UID: "0f3bc13c-e0e8-44a6-9885-5bcd094112ac") : object "kube-system"/"original-pull-secret" not registered Jul 28 12:06:23.915535 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:23.915472 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n podName:bd0bd704-8198-4017-a6d4-f8222009e54e nodeName:}" failed. No retries permitted until 2026-07-28 12:06:39.915464124 +0000 UTC m=+34.240129381 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6nt9n" (UniqueName: "kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n") pod "network-check-target-fd7d6" (UID: "bd0bd704-8198-4017-a6d4-f8222009e54e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 12:06:24.204290 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:24.204213 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:24.204484 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:24.204395 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:24.204484 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:24.204433 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:24.204484 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:24.204468 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:24.204657 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:24.204554 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:24.204657 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:24.204642 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:26.205441 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:26.205402 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:26.205894 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:26.205504 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:26.205894 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:26.205536 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:26.205894 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:26.205555 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:26.205894 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:26.205607 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:26.205894 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:26.205672 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:28.204424 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.204088 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:28.205143 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.204113 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:28.205143 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:28.204503 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:28.205143 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:28.204572 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:28.205143 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.204146 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:28.205143 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:28.204655 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:28.304555 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.304519 2575 generic.go:356] "Generic (PLEG): container finished" podID="7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425" containerID="1743bd550722e7a88816a86027528831463cc7957c5675e7938df97fb627da0a" exitCode=0 Jul 28 12:06:28.304714 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.304610 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2dw8" event={"ID":"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425","Type":"ContainerDied","Data":"1743bd550722e7a88816a86027528831463cc7957c5675e7938df97fb627da0a"} Jul 28 12:06:28.306491 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.306467 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d4d6q" event={"ID":"adc764ba-183f-42ff-b0b9-475dcda06640","Type":"ContainerStarted","Data":"8061cfd5ffee6c45159d9f8c6bd90253f44e054fb502752921509d261dd2d2c3"} Jul 28 12:06:28.308452 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.308427 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n97lp" event={"ID":"1c56eeda-c493-4055-87eb-1d2fea4a014d","Type":"ContainerStarted","Data":"12ed1bffa76f551136b1aafffcab7ee330b7638502b4700f5d8ce1f16f30f414"} Jul 28 12:06:28.311999 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.311957 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" event={"ID":"ee66aa6f-353c-464a-9a45-a009910fc6e3","Type":"ContainerStarted","Data":"5a17f0d74b63b9736534e58bb4316e652e479f9397bb614d4887649fa0d020fe"} Jul 28 12:06:28.312105 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.312052 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" event={"ID":"ee66aa6f-353c-464a-9a45-a009910fc6e3","Type":"ContainerStarted","Data":"ed558fe3c0161b53c6df9234c311e77e5d9aae94608fc9b9c4f1fe7c4a97f83a"} Jul 28 12:06:28.312168 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.312109 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" event={"ID":"ee66aa6f-353c-464a-9a45-a009910fc6e3","Type":"ContainerStarted","Data":"728c78cd402f808f0e11698855900d1840effc8362b84aa8e22df012d7627533"} Jul 28 12:06:28.312168 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.312157 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" event={"ID":"ee66aa6f-353c-464a-9a45-a009910fc6e3","Type":"ContainerStarted","Data":"d056e4fc11f2199b4ba1c372f0f5babcdb123cec3f215cec124643047e89516b"} Jul 28 12:06:28.312274 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.312205 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" event={"ID":"ee66aa6f-353c-464a-9a45-a009910fc6e3","Type":"ContainerStarted","Data":"b2922916f24ac24b96bd7c6fbe92ee6edff6535f300a8bc34a366da0669963ec"} Jul 28 12:06:28.312274 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.312252 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" event={"ID":"ee66aa6f-353c-464a-9a45-a009910fc6e3","Type":"ContainerStarted","Data":"bc42a1f4348d7362b41cf4e0b8baee601394e7891fab6b1ccdf1092785745636"} Jul 28 12:06:28.313509 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.313490 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" event={"ID":"4c4d1418-6b32-4534-b79c-5452b0d958d9","Type":"ContainerStarted","Data":"b62d84498e704a8cc5ca992dfcbe5e1bef490068cc03484203d93d2a105b731e"} Jul 28 12:06:28.315242 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.315083 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wnt8s" event={"ID":"20658b18-e0e0-4a39-a49a-7b386a1600cb","Type":"ContainerStarted","Data":"86ff70fa778f981fdabbf3d73701301d6e42055a73df7e39804c15f6279a706d"} Jul 28 12:06:28.316541 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.316518 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tzfpr" event={"ID":"00535017-b0ac-444e-ae72-711bf4003f21","Type":"ContainerStarted","Data":"86aceefe4511149148ec136e0c3c13b2a815ad4e11afa6a99e6374a168a80056"} Jul 28 12:06:28.318030 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.318010 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cml7q" event={"ID":"bd19a5ef-beec-427f-8a40-5c0f076a8c1d","Type":"ContainerStarted","Data":"d617cf67c8f6cf77e3f5d19274a6411aab2c61b7dae9a4627a12e482b8df0526"} Jul 28 12:06:28.332094 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.331995 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-158.ec2.internal" podStartSLOduration=21.331980442 podStartE2EDuration="21.331980442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:06:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:06:11.285928278 +0000 UTC m=+5.610593561" watchObservedRunningTime="2026-07-28 12:06:28.331980442 +0000 UTC m=+22.656645720" Jul 28 12:06:28.349223 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.349202 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tzfpr" Jul 28 12:06:28.349976 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.349958 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tzfpr" Jul 28 12:06:28.352089 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.352051 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cml7q" podStartSLOduration=3.923132888 podStartE2EDuration="22.352038813s" totalImagesPullingTime="18.428905925s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:06:28.351534278 +0000 UTC m=+22.676199555" watchObservedRunningTime="2026-07-28 12:06:28.352038813 +0000 UTC m=+22.676704093" Jul 28 12:06:28.396606 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.396569 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-n97lp" podStartSLOduration=3.98027859 podStartE2EDuration="22.396557836s" totalImagesPullingTime="18.416279246s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:06:28.373293994 +0000 UTC m=+22.697959368" watchObservedRunningTime="2026-07-28 12:06:28.396557836 +0000 UTC m=+22.721223113" Jul 28 12:06:28.396696 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.396649 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wnt8s" podStartSLOduration=3.9163191360000003 podStartE2EDuration="22.39664271s" totalImagesPullingTime="18.480323574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:06:28.396546997 +0000 UTC m=+22.721212276" watchObservedRunningTime="2026-07-28 12:06:28.39664271 +0000 UTC m=+22.721307987" Jul 28 12:06:28.430414 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.430379 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-d4d6q" podStartSLOduration=4.017078761 podStartE2EDuration="22.430368257s" totalImagesPullingTime="18.413289496s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:06:28.429954418 +0000 UTC m=+22.754619696" watchObservedRunningTime="2026-07-28 12:06:28.430368257 +0000 UTC m=+22.755033846" Jul 28 12:06:28.430666 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:28.430645 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tzfpr" podStartSLOduration=8.056573884 podStartE2EDuration="22.430638721s" totalImagesPullingTime="14.374064837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:06:28.414349898 +0000 UTC m=+22.739015176" watchObservedRunningTime="2026-07-28 12:06:28.430638721 +0000 UTC m=+22.755303999" Jul 28 12:06:29.077914 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:29.077887 2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 28 12:06:29.131766 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:29.131632 2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T12:06:29.077909975Z","UUID":"4f1162e7-0026-4797-a8e8-7c11c9c6ed9a","Handler":null,"Name":"","Endpoint":""} Jul 28 12:06:29.133101 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:29.133078 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 28 12:06:29.133216 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:29.133116 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 28 12:06:29.321604 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:29.321571 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-276l8" event={"ID":"5fe470f0-c27e-418f-af86-7fe9399cd0d4","Type":"ContainerStarted","Data":"c9bb6d5a2b2f14c81d0393e5c2b173153e715e502e5f8c25b7a600d6de794505"} Jul 28 12:06:29.323657 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:29.323572 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" event={"ID":"4c4d1418-6b32-4534-b79c-5452b0d958d9","Type":"ContainerStarted","Data":"34fcb4cfc3d13589c4425a50331af53a29324e9393260e5ad03d7c0ec5c144c1"} Jul 28 12:06:29.325876 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:29.325852 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tzfpr" Jul 28 12:06:29.325989 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:29.325935 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tzfpr" Jul 28 12:06:29.363139 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:29.363091 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-276l8" podStartSLOduration=4.941861519 podStartE2EDuration="23.363075696s" totalImagesPullingTime="18.421214177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:06:29.361928839 +0000 UTC m=+23.686594118" watchObservedRunningTime="2026-07-28 12:06:29.363075696 +0000 UTC m=+23.687740974" Jul 28 12:06:30.204176 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:30.204148 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:30.204286 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:30.204149 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:30.204286 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:30.204149 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:30.204286 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:30.204258 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:30.204476 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:30.204352 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:30.204476 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:30.204423 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:30.328592 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:30.328538 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" event={"ID":"4c4d1418-6b32-4534-b79c-5452b0d958d9","Type":"ContainerStarted","Data":"c789a899c90762aea3169c511814350fd97b53e61384b60b522dfc1055586c63"} Jul 28 12:06:30.355057 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:30.355007 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9hhjl" podStartSLOduration=3.715082685 podStartE2EDuration="24.354986109s" totalImagesPullingTime="20.639903424s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:06:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:06:30.354196426 +0000 UTC m=+24.678861705" watchObservedRunningTime="2026-07-28 12:06:30.354986109 +0000 UTC m=+24.679651390" Jul 28 12:06:31.334618 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:31.334580 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" event={"ID":"ee66aa6f-353c-464a-9a45-a009910fc6e3","Type":"ContainerStarted","Data":"d7636a2228d0e47eb806d279e223979a9778d9439b0774ea7506d89be9c722d2"} Jul 28 12:06:32.204065 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:32.204029 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:32.204065 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:32.204064 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:32.204319 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:32.204038 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:32.204319 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:32.204161 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:32.204319 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:32.204285 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:32.204463 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:32.204414 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:33.342391 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:33.342151 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" event={"ID":"ee66aa6f-353c-464a-9a45-a009910fc6e3","Type":"ContainerStarted","Data":"56e6a3bf5c9479daac4ee3fe62d6d70a369d21772f9e6a00c7580857e537e438"} Jul 28 12:06:33.343157 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:33.342500 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:33.343157 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:33.342556 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:33.344212 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:33.344184 2575 generic.go:356] "Generic (PLEG): container finished" podID="7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425" containerID="380a6ba5db131cec992bf74dc4758b8bd41a18c44783da25253c084124635217" exitCode=0 Jul 28 12:06:33.344440 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:33.344223 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2dw8" event={"ID":"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425","Type":"ContainerDied","Data":"380a6ba5db131cec992bf74dc4758b8bd41a18c44783da25253c084124635217"} Jul 28 12:06:33.358179 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:33.358153 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:33.373294 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:33.373252 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" podStartSLOduration=8.886819687 podStartE2EDuration="27.373239835s" totalImagesPullingTime="18.486420148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:06:33.37270309 +0000 UTC m=+27.697368367" watchObservedRunningTime="2026-07-28 12:06:33.373239835 +0000 UTC m=+27.697905153" Jul 28 12:06:34.204227 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:34.204059 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:34.204456 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:34.204063 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:34.204456 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:34.204323 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:34.204456 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:34.204079 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:34.204630 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:34.204460 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:34.204630 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:34.204562 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:34.352574 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:34.352547 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:34.366166 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:34.366145 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:06:34.754642 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:34.754617 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5f72v"] Jul 28 12:06:34.754794 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:34.754746 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:34.754881 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:34.754860 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:34.755129 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:34.755107 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fd7d6"] Jul 28 12:06:34.755198 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:34.755183 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:34.755277 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:34.755242 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:34.756272 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:34.756251 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kkbtd"] Jul 28 12:06:34.756394 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:34.756350 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:34.756434 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:34.756417 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:35.356300 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:35.356266 2575 generic.go:356] "Generic (PLEG): container finished" podID="7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425" containerID="b08a35624932763f0e945b431d6af0b826e94c0c7be2e9e66027b12bc04b71b6" exitCode=0 Jul 28 12:06:35.356685 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:35.356358 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2dw8" event={"ID":"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425","Type":"ContainerDied","Data":"b08a35624932763f0e945b431d6af0b826e94c0c7be2e9e66027b12bc04b71b6"} Jul 28 12:06:36.205589 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:36.205556 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:36.205756 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:36.205664 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:36.205756 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:36.205691 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:36.205756 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:36.205736 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:37.203769 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:37.203737 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:37.204214 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:37.203870 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:37.362620 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:37.362464 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2dw8" event={"ID":"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425","Type":"ContainerStarted","Data":"010eafa2a404dd502e375269de5ac24bf30a32d6880e27a5e6cc05883ccc7d3f"} Jul 28 12:06:38.204648 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:38.204614 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:38.205054 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:38.204722 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:38.205054 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:38.204793 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:38.205054 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:38.204914 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:38.367285 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:38.367252 2575 generic.go:356] "Generic (PLEG): container finished" podID="7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425" containerID="010eafa2a404dd502e375269de5ac24bf30a32d6880e27a5e6cc05883ccc7d3f" exitCode=0 Jul 28 12:06:38.367428 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:38.367316 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2dw8" event={"ID":"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425","Type":"ContainerDied","Data":"010eafa2a404dd502e375269de5ac24bf30a32d6880e27a5e6cc05883ccc7d3f"} Jul 28 12:06:39.204427 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:39.204400 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:39.204583 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:39.204495 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:39.834319 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:39.834283 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs\") pod \"network-metrics-daemon-5f72v\" (UID: \"92daaf51-12ba-4eab-87ac-2087d60aac39\") " pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:39.834730 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:39.834403 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 12:06:39.834730 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:39.834467 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs podName:92daaf51-12ba-4eab-87ac-2087d60aac39 nodeName:}" failed. No retries permitted until 2026-07-28 12:07:11.834445541 +0000 UTC m=+66.159110801 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs") pod "network-metrics-daemon-5f72v" (UID: "92daaf51-12ba-4eab-87ac-2087d60aac39") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 12:06:39.934573 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:39.934544 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nt9n\" (UniqueName: \"kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n\") pod \"network-check-target-fd7d6\" (UID: \"bd0bd704-8198-4017-a6d4-f8222009e54e\") " pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:39.934697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:39.934601 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret\") pod \"global-pull-secret-syncer-kkbtd\" (UID: \"0f3bc13c-e0e8-44a6-9885-5bcd094112ac\") " pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:39.934697 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:39.934676 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 12:06:39.934766 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:39.934695 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 12:06:39.934766 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:39.934713 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 12:06:39.934766 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:39.934718 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret podName:0f3bc13c-e0e8-44a6-9885-5bcd094112ac nodeName:}" failed. No retries permitted until 2026-07-28 12:07:11.934706261 +0000 UTC m=+66.259371516 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret") pod "global-pull-secret-syncer-kkbtd" (UID: "0f3bc13c-e0e8-44a6-9885-5bcd094112ac") : object "kube-system"/"original-pull-secret" not registered Jul 28 12:06:39.934766 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:39.934723 2575 projected.go:196] Error preparing data for projected volume kube-api-access-6nt9n for pod openshift-network-diagnostics/network-check-target-fd7d6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 12:06:39.934923 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:39.934767 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n podName:bd0bd704-8198-4017-a6d4-f8222009e54e nodeName:}" failed. No retries permitted until 2026-07-28 12:07:11.934755087 +0000 UTC m=+66.259420346 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6nt9n" (UniqueName: "kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n") pod "network-check-target-fd7d6" (UID: "bd0bd704-8198-4017-a6d4-f8222009e54e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 12:06:40.203974 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:40.203943 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:40.204146 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:40.204075 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5f72v" podUID="92daaf51-12ba-4eab-87ac-2087d60aac39" Jul 28 12:06:40.204146 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:40.204131 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:40.204244 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:40.204229 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fd7d6" podUID="bd0bd704-8198-4017-a6d4-f8222009e54e" Jul 28 12:06:41.204471 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.204435 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:41.204939 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:41.204541 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkbtd" podUID="0f3bc13c-e0e8-44a6-9885-5bcd094112ac" Jul 28 12:06:41.526543 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.526515 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-158.ec2.internal" event="NodeReady" Jul 28 12:06:41.526731 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.526649 2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 28 12:06:41.582647 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.582617 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6k26v"] Jul 28 12:06:41.585099 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.585082 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:41.586519 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.586495 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-x4qcv"] Jul 28 12:06:41.587862 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.587841 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 28 12:06:41.587985 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.587901 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4j8p8\"" Jul 28 12:06:41.587985 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.587904 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 28 12:06:41.588272 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.588252 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x4qcv" Jul 28 12:06:41.592380 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.592362 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-64w8m\"" Jul 28 12:06:41.592545 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.592523 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 28 12:06:41.592936 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.592684 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 28 12:06:41.593268 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.593246 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 28 12:06:41.597023 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.597006 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6k26v"] Jul 28 12:06:41.602296 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.602276 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x4qcv"] Jul 28 12:06:41.747942 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.747890 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zpmm\" (UniqueName: \"kubernetes.io/projected/557e6458-36ad-4c2d-bc32-ba9540692a3f-kube-api-access-5zpmm\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:41.747942 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.747937 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdxmg\" (UniqueName: \"kubernetes.io/projected/1974e756-46ac-42da-886f-8353ebd17e41-kube-api-access-wdxmg\") pod \"ingress-canary-x4qcv\" (UID: \"1974e756-46ac-42da-886f-8353ebd17e41\") " pod="openshift-ingress-canary/ingress-canary-x4qcv" Jul 28 12:06:41.748188 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.748014 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/557e6458-36ad-4c2d-bc32-ba9540692a3f-config-volume\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:41.748188 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.748043 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1974e756-46ac-42da-886f-8353ebd17e41-cert\") pod \"ingress-canary-x4qcv\" (UID: \"1974e756-46ac-42da-886f-8353ebd17e41\") " pod="openshift-ingress-canary/ingress-canary-x4qcv" Jul 28 12:06:41.748188 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.748066 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:41.748188 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.748124 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/557e6458-36ad-4c2d-bc32-ba9540692a3f-tmp-dir\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:41.849248 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.849168 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zpmm\" (UniqueName: \"kubernetes.io/projected/557e6458-36ad-4c2d-bc32-ba9540692a3f-kube-api-access-5zpmm\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:41.849248 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.849213 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdxmg\" (UniqueName: \"kubernetes.io/projected/1974e756-46ac-42da-886f-8353ebd17e41-kube-api-access-wdxmg\") pod \"ingress-canary-x4qcv\" (UID: \"1974e756-46ac-42da-886f-8353ebd17e41\") " pod="openshift-ingress-canary/ingress-canary-x4qcv" Jul 28 12:06:41.849480 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.849322 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/557e6458-36ad-4c2d-bc32-ba9540692a3f-config-volume\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:41.849480 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.849354 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1974e756-46ac-42da-886f-8353ebd17e41-cert\") pod \"ingress-canary-x4qcv\" (UID: \"1974e756-46ac-42da-886f-8353ebd17e41\") " pod="openshift-ingress-canary/ingress-canary-x4qcv" Jul 28 12:06:41.849480 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.849391 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:41.849480 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.849439 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/557e6458-36ad-4c2d-bc32-ba9540692a3f-tmp-dir\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:41.849671 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:41.849553 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 12:06:41.849671 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:41.849619 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls podName:557e6458-36ad-4c2d-bc32-ba9540692a3f nodeName:}" failed. No retries permitted until 2026-07-28 12:06:42.349596002 +0000 UTC m=+36.674261273 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls") pod "dns-default-6k26v" (UID: "557e6458-36ad-4c2d-bc32-ba9540692a3f") : secret "dns-default-metrics-tls" not found Jul 28 12:06:41.849848 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.849823 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/557e6458-36ad-4c2d-bc32-ba9540692a3f-tmp-dir\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:41.849985 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.849968 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/557e6458-36ad-4c2d-bc32-ba9540692a3f-config-volume\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:41.854705 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.854666 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1974e756-46ac-42da-886f-8353ebd17e41-cert\") pod \"ingress-canary-x4qcv\" (UID: \"1974e756-46ac-42da-886f-8353ebd17e41\") " pod="openshift-ingress-canary/ingress-canary-x4qcv" Jul 28 12:06:41.860967 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.860926 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdxmg\" (UniqueName: \"kubernetes.io/projected/1974e756-46ac-42da-886f-8353ebd17e41-kube-api-access-wdxmg\") pod \"ingress-canary-x4qcv\" (UID: \"1974e756-46ac-42da-886f-8353ebd17e41\") " pod="openshift-ingress-canary/ingress-canary-x4qcv" Jul 28 12:06:41.861174 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.861152 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zpmm\" (UniqueName: \"kubernetes.io/projected/557e6458-36ad-4c2d-bc32-ba9540692a3f-kube-api-access-5zpmm\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:41.903363 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:41.903329 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x4qcv" Jul 28 12:06:42.204009 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:42.203977 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:06:42.204009 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:42.203993 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:06:42.208247 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:42.208221 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qjs5z\"" Jul 28 12:06:42.210436 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:42.210034 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n7r5r\"" Jul 28 12:06:42.210436 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:42.210075 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 12:06:42.210436 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:42.210179 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 12:06:42.211675 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:42.211596 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 12:06:42.353819 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:42.353782 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:42.353978 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:42.353916 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 12:06:42.353978 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:42.353973 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls podName:557e6458-36ad-4c2d-bc32-ba9540692a3f nodeName:}" failed. No retries permitted until 2026-07-28 12:06:43.353957885 +0000 UTC m=+37.678623140 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls") pod "dns-default-6k26v" (UID: "557e6458-36ad-4c2d-bc32-ba9540692a3f") : secret "dns-default-metrics-tls" not found Jul 28 12:06:42.697127 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:42.696862 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x4qcv"] Jul 28 12:06:42.699553 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:42.699531 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1974e756_46ac_42da_886f_8353ebd17e41.slice/crio-764c0df780c30970edf324deea0aa60f062d9a9748fc15fa76a4f1f7d086da17 WatchSource:0}: Error finding container 764c0df780c30970edf324deea0aa60f062d9a9748fc15fa76a4f1f7d086da17: Status 404 returned error can't find the container with id 764c0df780c30970edf324deea0aa60f062d9a9748fc15fa76a4f1f7d086da17 Jul 28 12:06:43.203739 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:43.203714 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:06:43.206806 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:43.206783 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 12:06:43.362028 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:43.362002 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:43.362650 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:43.362158 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 12:06:43.362650 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:43.362239 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls podName:557e6458-36ad-4c2d-bc32-ba9540692a3f nodeName:}" failed. No retries permitted until 2026-07-28 12:06:45.36221984 +0000 UTC m=+39.686885118 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls") pod "dns-default-6k26v" (UID: "557e6458-36ad-4c2d-bc32-ba9540692a3f") : secret "dns-default-metrics-tls" not found Jul 28 12:06:43.380667 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:43.380637 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x4qcv" event={"ID":"1974e756-46ac-42da-886f-8353ebd17e41","Type":"ContainerStarted","Data":"764c0df780c30970edf324deea0aa60f062d9a9748fc15fa76a4f1f7d086da17"} Jul 28 12:06:43.384469 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:43.384441 2575 generic.go:356] "Generic (PLEG): container finished" podID="7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425" containerID="09d677b77d1ab5b3b10560d67a0487df0ca00a5a01f06c414410cfd3a11d34da" exitCode=0 Jul 28 12:06:43.384579 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:43.384501 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2dw8" event={"ID":"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425","Type":"ContainerDied","Data":"09d677b77d1ab5b3b10560d67a0487df0ca00a5a01f06c414410cfd3a11d34da"} Jul 28 12:06:44.390057 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:44.390022 2575 generic.go:356] "Generic (PLEG): container finished" podID="7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425" containerID="3a0f12f96baacad3dc8c0ef20db6228dc17c0e138186b76c0494832e176498c1" exitCode=0 Jul 28 12:06:44.390482 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:44.390087 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2dw8" event={"ID":"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425","Type":"ContainerDied","Data":"3a0f12f96baacad3dc8c0ef20db6228dc17c0e138186b76c0494832e176498c1"} Jul 28 12:06:45.377898 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:45.377865 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:45.378053 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:45.377990 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 12:06:45.378053 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:45.378042 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls podName:557e6458-36ad-4c2d-bc32-ba9540692a3f nodeName:}" failed. No retries permitted until 2026-07-28 12:06:49.378028635 +0000 UTC m=+43.702693891 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls") pod "dns-default-6k26v" (UID: "557e6458-36ad-4c2d-bc32-ba9540692a3f") : secret "dns-default-metrics-tls" not found Jul 28 12:06:45.396251 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:45.396224 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q2dw8" event={"ID":"7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425","Type":"ContainerStarted","Data":"b8ee32c534e9f17a7aec8f701d0e79358a30c8fe77ecce37ac1abbd4d5fc0e48"} Jul 28 12:06:45.424130 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:45.424085 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-q2dw8" podStartSLOduration=9.547660041 podStartE2EDuration="39.424072642s" totalImagesPullingTime="29.876412601s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:06:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:06:45.420937618 +0000 UTC m=+39.745602897" watchObservedRunningTime="2026-07-28 12:06:45.424072642 +0000 UTC m=+39.748737897" Jul 28 12:06:46.065169 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.065138 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz"] Jul 28 12:06:46.081726 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.081702 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz"] Jul 28 12:06:46.081862 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.081790 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:06:46.084729 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.084709 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 28 12:06:46.084961 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.084754 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kmjbh\"" Jul 28 12:06:46.087126 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.087105 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 28 12:06:46.087239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.087177 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 28 12:06:46.087239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.087195 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 28 12:06:46.171907 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.171887 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h"] Jul 28 12:06:46.183252 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.183231 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwqrl\" (UniqueName: \"kubernetes.io/projected/cff40866-ec36-4069-99af-4409e6f10e55-kube-api-access-lwqrl\") pod \"cluster-monitoring-operator-548b489cb8-dn6dz\" (UID: \"cff40866-ec36-4069-99af-4409e6f10e55\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:06:46.183342 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.183284 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cff40866-ec36-4069-99af-4409e6f10e55-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-dn6dz\" (UID: \"cff40866-ec36-4069-99af-4409e6f10e55\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:06:46.183342 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.183327 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dn6dz\" (UID: \"cff40866-ec36-4069-99af-4409e6f10e55\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:06:46.195595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.195571 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h" Jul 28 12:06:46.195595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.195573 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h"] Jul 28 12:06:46.195735 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.195623 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-hk9cz"] Jul 28 12:06:46.200363 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.200345 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 28 12:06:46.200446 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.200403 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 28 12:06:46.201637 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.201614 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 28 12:06:46.201828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.201816 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 28 12:06:46.202165 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.202152 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6chqw\"" Jul 28 12:06:46.215699 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.215676 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hk9cz" Jul 28 12:06:46.215777 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.215703 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-hk9cz"] Jul 28 12:06:46.219146 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.219131 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cbjzq\"" Jul 28 12:06:46.275981 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.275964 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6zc47"] Jul 28 12:06:46.284353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.284331 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qccsr\" (UniqueName: \"kubernetes.io/projected/0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7-kube-api-access-qccsr\") pod \"service-ca-operator-5d6c75f84f-xp72h\" (UID: \"0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h" Jul 28 12:06:46.284994 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.284647 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cff40866-ec36-4069-99af-4409e6f10e55-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-dn6dz\" (UID: \"cff40866-ec36-4069-99af-4409e6f10e55\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:06:46.284994 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.284824 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dn6dz\" (UID: \"cff40866-ec36-4069-99af-4409e6f10e55\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:06:46.284994 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.284894 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7-serving-cert\") pod \"service-ca-operator-5d6c75f84f-xp72h\" (UID: \"0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h" Jul 28 12:06:46.284994 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.284955 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwqrl\" (UniqueName: \"kubernetes.io/projected/cff40866-ec36-4069-99af-4409e6f10e55-kube-api-access-lwqrl\") pod \"cluster-monitoring-operator-548b489cb8-dn6dz\" (UID: \"cff40866-ec36-4069-99af-4409e6f10e55\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:06:46.285240 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:46.285084 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 12:06:46.287771 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.285455 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7-config\") pod \"service-ca-operator-5d6c75f84f-xp72h\" (UID: \"0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h" Jul 28 12:06:46.287771 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.285616 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cff40866-ec36-4069-99af-4409e6f10e55-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-dn6dz\" (UID: \"cff40866-ec36-4069-99af-4409e6f10e55\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:06:46.287771 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:46.285758 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls podName:cff40866-ec36-4069-99af-4409e6f10e55 nodeName:}" failed. No retries permitted until 2026-07-28 12:06:46.785728508 +0000 UTC m=+41.110393765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dn6dz" (UID: "cff40866-ec36-4069-99af-4409e6f10e55") : secret "cluster-monitoring-operator-tls" not found Jul 28 12:06:46.293592 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.293570 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-684fd7d5f9-8xm5z"] Jul 28 12:06:46.293699 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.293683 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6zc47" Jul 28 12:06:46.296757 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.296736 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-jqr7m\"" Jul 28 12:06:46.296831 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.296757 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 28 12:06:46.297351 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.297328 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 28 12:06:46.309220 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.309203 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwqrl\" (UniqueName: \"kubernetes.io/projected/cff40866-ec36-4069-99af-4409e6f10e55-kube-api-access-lwqrl\") pod \"cluster-monitoring-operator-548b489cb8-dn6dz\" (UID: \"cff40866-ec36-4069-99af-4409e6f10e55\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:06:46.314079 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.314060 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-l4pfh"] Jul 28 12:06:46.314196 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.314179 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:46.318677 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.318118 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 28 12:06:46.318677 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.318234 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 28 12:06:46.318677 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.318511 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 28 12:06:46.318677 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.318558 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 28 12:06:46.318677 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.318590 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 28 12:06:46.319356 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.318831 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 28 12:06:46.319522 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.319161 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kt62m\"" Jul 28 12:06:46.328127 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.328110 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6zc47"] Jul 28 12:06:46.328192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.328136 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-684fd7d5f9-8xm5z"] Jul 28 12:06:46.328192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.328150 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-l4pfh"] Jul 28 12:06:46.328264 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.328206 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:06:46.331977 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.331959 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 28 12:06:46.332292 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.332278 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 28 12:06:46.332405 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.332372 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jqtwl\"" Jul 28 12:06:46.332673 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.332654 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 28 12:06:46.332764 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.332707 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 28 12:06:46.338186 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.338164 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 28 12:06:46.386505 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.386481 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59fh7\" (UniqueName: \"kubernetes.io/projected/804fcdbc-1bb1-4ca3-9e40-e5ab14d8daad-kube-api-access-59fh7\") pod \"network-check-source-7d88cb55d8-hk9cz\" (UID: \"804fcdbc-1bb1-4ca3-9e40-e5ab14d8daad\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hk9cz" Jul 28 12:06:46.386595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.386520 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7-serving-cert\") pod \"service-ca-operator-5d6c75f84f-xp72h\" (UID: \"0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h" Jul 28 12:06:46.386595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.386548 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7-config\") pod \"service-ca-operator-5d6c75f84f-xp72h\" (UID: \"0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h" Jul 28 12:06:46.386595 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.386585 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qccsr\" (UniqueName: \"kubernetes.io/projected/0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7-kube-api-access-qccsr\") pod \"service-ca-operator-5d6c75f84f-xp72h\" (UID: \"0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h" Jul 28 12:06:46.386728 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.386672 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6s2t\" (UniqueName: \"kubernetes.io/projected/1c2bc53b-e91d-40c4-a177-3724af9c37e9-kube-api-access-v6s2t\") pod \"volume-data-source-validator-549c6d54cf-6zc47\" (UID: \"1c2bc53b-e91d-40c4-a177-3724af9c37e9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6zc47" Jul 28 12:06:46.387059 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.387042 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7-config\") pod \"service-ca-operator-5d6c75f84f-xp72h\" (UID: \"0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h" Jul 28 12:06:46.388690 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.388674 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7-serving-cert\") pod \"service-ca-operator-5d6c75f84f-xp72h\" (UID: \"0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h" Jul 28 12:06:46.395187 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.395166 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qccsr\" (UniqueName: \"kubernetes.io/projected/0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7-kube-api-access-qccsr\") pod \"service-ca-operator-5d6c75f84f-xp72h\" (UID: \"0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h" Jul 28 12:06:46.400745 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.400725 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x4qcv" event={"ID":"1974e756-46ac-42da-886f-8353ebd17e41","Type":"ContainerStarted","Data":"bd304479d31d5bdcb6fc08d208dd1023fa327091c75559be2c3f9ddb72d401e1"} Jul 28 12:06:46.420274 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.420235 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-x4qcv" podStartSLOduration=2.5246517920000002 podStartE2EDuration="5.420223987s" totalImagesPullingTime="2.895572195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:06:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:06:46.420168763 +0000 UTC m=+40.744834041" watchObservedRunningTime="2026-07-28 12:06:46.420223987 +0000 UTC m=+40.744889243" Jul 28 12:06:46.487581 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.487555 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3dc2810f-7c06-49ad-b0d6-d87a52fd6317-config\") pod \"console-operator-584dcb6b54-l4pfh\" (UID: \"3dc2810f-7c06-49ad-b0d6-d87a52fd6317\") " pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:06:46.487664 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.487584 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3dc2810f-7c06-49ad-b0d6-d87a52fd6317-serving-cert\") pod \"console-operator-584dcb6b54-l4pfh\" (UID: \"3dc2810f-7c06-49ad-b0d6-d87a52fd6317\") " pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:06:46.487664 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.487618 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htbx7\" (UniqueName: \"kubernetes.io/projected/3dc2810f-7c06-49ad-b0d6-d87a52fd6317-kube-api-access-htbx7\") pod \"console-operator-584dcb6b54-l4pfh\" (UID: \"3dc2810f-7c06-49ad-b0d6-d87a52fd6317\") " pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:06:46.487664 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.487658 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6s2t\" (UniqueName: \"kubernetes.io/projected/1c2bc53b-e91d-40c4-a177-3724af9c37e9-kube-api-access-v6s2t\") pod \"volume-data-source-validator-549c6d54cf-6zc47\" (UID: \"1c2bc53b-e91d-40c4-a177-3724af9c37e9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6zc47" Jul 28 12:06:46.487808 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.487688 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-default-certificate\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:46.487808 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.487702 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:46.487808 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.487725 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59fh7\" (UniqueName: \"kubernetes.io/projected/804fcdbc-1bb1-4ca3-9e40-e5ab14d8daad-kube-api-access-59fh7\") pod \"network-check-source-7d88cb55d8-hk9cz\" (UID: \"804fcdbc-1bb1-4ca3-9e40-e5ab14d8daad\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hk9cz" Jul 28 12:06:46.487808 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.487741 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:46.487808 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.487760 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v62np\" (UniqueName: \"kubernetes.io/projected/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-kube-api-access-v62np\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:46.487808 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.487776 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3dc2810f-7c06-49ad-b0d6-d87a52fd6317-trusted-ca\") pod \"console-operator-584dcb6b54-l4pfh\" (UID: \"3dc2810f-7c06-49ad-b0d6-d87a52fd6317\") " pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:06:46.488042 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.487864 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-stats-auth\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:46.496413 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.496396 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59fh7\" (UniqueName: \"kubernetes.io/projected/804fcdbc-1bb1-4ca3-9e40-e5ab14d8daad-kube-api-access-59fh7\") pod \"network-check-source-7d88cb55d8-hk9cz\" (UID: \"804fcdbc-1bb1-4ca3-9e40-e5ab14d8daad\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hk9cz" Jul 28 12:06:46.496482 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.496439 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6s2t\" (UniqueName: \"kubernetes.io/projected/1c2bc53b-e91d-40c4-a177-3724af9c37e9-kube-api-access-v6s2t\") pod \"volume-data-source-validator-549c6d54cf-6zc47\" (UID: \"1c2bc53b-e91d-40c4-a177-3724af9c37e9\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6zc47" Jul 28 12:06:46.503352 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.503330 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h" Jul 28 12:06:46.524568 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.524549 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hk9cz" Jul 28 12:06:46.589291 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.589260 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-stats-auth\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:46.589427 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.589345 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3dc2810f-7c06-49ad-b0d6-d87a52fd6317-config\") pod \"console-operator-584dcb6b54-l4pfh\" (UID: \"3dc2810f-7c06-49ad-b0d6-d87a52fd6317\") " pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:06:46.589427 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.589396 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3dc2810f-7c06-49ad-b0d6-d87a52fd6317-serving-cert\") pod \"console-operator-584dcb6b54-l4pfh\" (UID: \"3dc2810f-7c06-49ad-b0d6-d87a52fd6317\") " pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:06:46.589548 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.589453 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htbx7\" (UniqueName: \"kubernetes.io/projected/3dc2810f-7c06-49ad-b0d6-d87a52fd6317-kube-api-access-htbx7\") pod \"console-operator-584dcb6b54-l4pfh\" (UID: \"3dc2810f-7c06-49ad-b0d6-d87a52fd6317\") " pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:06:46.589548 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.589538 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-default-certificate\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:46.589643 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.589562 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:46.589643 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.589605 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:46.590070 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:46.589878 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 12:06:46.590070 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:46.589946 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs podName:e2d43bb6-5482-4950-b2a5-b06056f4fa3c nodeName:}" failed. No retries permitted until 2026-07-28 12:06:47.08992403 +0000 UTC m=+41.414589303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs") pod "router-default-684fd7d5f9-8xm5z" (UID: "e2d43bb6-5482-4950-b2a5-b06056f4fa3c") : secret "router-metrics-certs-default" not found Jul 28 12:06:46.590232 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:46.590140 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle podName:e2d43bb6-5482-4950-b2a5-b06056f4fa3c nodeName:}" failed. No retries permitted until 2026-07-28 12:06:47.090124805 +0000 UTC m=+41.414790062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle") pod "router-default-684fd7d5f9-8xm5z" (UID: "e2d43bb6-5482-4950-b2a5-b06056f4fa3c") : configmap references non-existent config key: service-ca.crt Jul 28 12:06:46.590281 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.590236 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v62np\" (UniqueName: \"kubernetes.io/projected/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-kube-api-access-v62np\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:46.590281 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.590261 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3dc2810f-7c06-49ad-b0d6-d87a52fd6317-trusted-ca\") pod \"console-operator-584dcb6b54-l4pfh\" (UID: \"3dc2810f-7c06-49ad-b0d6-d87a52fd6317\") " pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:06:46.590784 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.590695 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3dc2810f-7c06-49ad-b0d6-d87a52fd6317-config\") pod \"console-operator-584dcb6b54-l4pfh\" (UID: \"3dc2810f-7c06-49ad-b0d6-d87a52fd6317\") " pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:06:46.593295 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.591293 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3dc2810f-7c06-49ad-b0d6-d87a52fd6317-trusted-ca\") pod \"console-operator-584dcb6b54-l4pfh\" (UID: \"3dc2810f-7c06-49ad-b0d6-d87a52fd6317\") " pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:06:46.593295 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.592770 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3dc2810f-7c06-49ad-b0d6-d87a52fd6317-serving-cert\") pod \"console-operator-584dcb6b54-l4pfh\" (UID: \"3dc2810f-7c06-49ad-b0d6-d87a52fd6317\") " pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:06:46.593295 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.593213 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-stats-auth\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:46.594076 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.593997 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-default-certificate\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:46.600409 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.600237 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htbx7\" (UniqueName: \"kubernetes.io/projected/3dc2810f-7c06-49ad-b0d6-d87a52fd6317-kube-api-access-htbx7\") pod \"console-operator-584dcb6b54-l4pfh\" (UID: \"3dc2810f-7c06-49ad-b0d6-d87a52fd6317\") " pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:06:46.602076 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.602051 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v62np\" (UniqueName: \"kubernetes.io/projected/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-kube-api-access-v62np\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:46.603984 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.603575 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6zc47" Jul 28 12:06:46.636458 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.636425 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:06:46.672733 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.672704 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h"] Jul 28 12:06:46.675820 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:46.675788 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b4caacf_830c_46b5_8b7a_5d0b06a9f7c7.slice/crio-8c1da7d6c9dae1f03aad06696a791c6bf733505717a43ea6d50d6a6785836b7e WatchSource:0}: Error finding container 8c1da7d6c9dae1f03aad06696a791c6bf733505717a43ea6d50d6a6785836b7e: Status 404 returned error can't find the container with id 8c1da7d6c9dae1f03aad06696a791c6bf733505717a43ea6d50d6a6785836b7e Jul 28 12:06:46.699943 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.699910 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-hk9cz"] Jul 28 12:06:46.714925 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:46.711762 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod804fcdbc_1bb1_4ca3_9e40_e5ab14d8daad.slice/crio-b07c02dd0fe649c1fbeb25fc79a577abc51610c32fe9cde96fff24fc6cd51cec WatchSource:0}: Error finding container b07c02dd0fe649c1fbeb25fc79a577abc51610c32fe9cde96fff24fc6cd51cec: Status 404 returned error can't find the container with id b07c02dd0fe649c1fbeb25fc79a577abc51610c32fe9cde96fff24fc6cd51cec Jul 28 12:06:46.779342 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.779301 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-844d6895c9-9l962"] Jul 28 12:06:46.794274 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.794247 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dn6dz\" (UID: \"cff40866-ec36-4069-99af-4409e6f10e55\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:06:46.794406 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:46.794392 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 12:06:46.794460 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:46.794449 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls podName:cff40866-ec36-4069-99af-4409e6f10e55 nodeName:}" failed. No retries permitted until 2026-07-28 12:06:47.794434071 +0000 UTC m=+42.119099327 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dn6dz" (UID: "cff40866-ec36-4069-99af-4409e6f10e55") : secret "cluster-monitoring-operator-tls" not found Jul 28 12:06:46.804634 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:46.804611 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c2bc53b_e91d_40c4_a177_3724af9c37e9.slice/crio-86fb88376ffc3a8720e12d96da8accf423511cedc9b290f857d3910af56d60bd WatchSource:0}: Error finding container 86fb88376ffc3a8720e12d96da8accf423511cedc9b290f857d3910af56d60bd: Status 404 returned error can't find the container with id 86fb88376ffc3a8720e12d96da8accf423511cedc9b290f857d3910af56d60bd Jul 28 12:06:46.807609 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.807581 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-844d6895c9-9l962"] Jul 28 12:06:46.807668 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.807621 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6zc47"] Jul 28 12:06:46.807715 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.807686 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:46.810229 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.810207 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 28 12:06:46.810229 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.810218 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 28 12:06:46.810410 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.810265 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8rhtj\"" Jul 28 12:06:46.810410 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.810284 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 28 12:06:46.817596 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.817522 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 28 12:06:46.819298 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.819277 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-l4pfh"] Jul 28 12:06:46.822631 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:06:46.822605 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dc2810f_7c06_49ad_b0d6_d87a52fd6317.slice/crio-18dfa375960d5ca627afe29f628619f8425601da34b0cac8c510aa71f18f1f00 WatchSource:0}: Error finding container 18dfa375960d5ca627afe29f628619f8425601da34b0cac8c510aa71f18f1f00: Status 404 returned error can't find the container with id 18dfa375960d5ca627afe29f628619f8425601da34b0cac8c510aa71f18f1f00 Jul 28 12:06:46.996232 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.996208 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-trusted-ca\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:46.996327 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.996241 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-installation-pull-secrets\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:46.996327 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.996258 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qdk6\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-kube-api-access-7qdk6\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:46.996403 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.996378 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-image-registry-private-configuration\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:46.996439 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.996411 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-bound-sa-token\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:46.996469 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.996459 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:46.996497 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.996477 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-ca-trust-extracted\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:46.996497 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:46.996493 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-certificates\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.097328 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.097247 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-bound-sa-token\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.097328 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.097281 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.097328 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.097298 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-ca-trust-extracted\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.097576 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.097331 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-certificates\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.097576 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:47.097427 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 12:06:47.097576 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:47.097444 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-844d6895c9-9l962: secret "image-registry-tls" not found Jul 28 12:06:47.097576 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.097449 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:47.097576 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:47.097494 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls podName:b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0 nodeName:}" failed. No retries permitted until 2026-07-28 12:06:47.59747583 +0000 UTC m=+41.922141089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls") pod "image-registry-844d6895c9-9l962" (UID: "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0") : secret "image-registry-tls" not found Jul 28 12:06:47.097576 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:47.097547 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle podName:e2d43bb6-5482-4950-b2a5-b06056f4fa3c nodeName:}" failed. No retries permitted until 2026-07-28 12:06:48.097530735 +0000 UTC m=+42.422195995 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle") pod "router-default-684fd7d5f9-8xm5z" (UID: "e2d43bb6-5482-4950-b2a5-b06056f4fa3c") : configmap references non-existent config key: service-ca.crt Jul 28 12:06:47.097803 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.097590 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-trusted-ca\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.097803 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.097627 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-installation-pull-secrets\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.097803 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.097656 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qdk6\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-kube-api-access-7qdk6\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.097803 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.097685 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-ca-trust-extracted\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.097803 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.097788 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:47.098028 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.097849 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-image-registry-private-configuration\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.098028 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.097912 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-certificates\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.098028 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:47.097935 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 12:06:47.098028 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:47.097998 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs podName:e2d43bb6-5482-4950-b2a5-b06056f4fa3c nodeName:}" failed. No retries permitted until 2026-07-28 12:06:48.09797274 +0000 UTC m=+42.422638009 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs") pod "router-default-684fd7d5f9-8xm5z" (UID: "e2d43bb6-5482-4950-b2a5-b06056f4fa3c") : secret "router-metrics-certs-default" not found Jul 28 12:06:47.109290 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.109269 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-installation-pull-secrets\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.109397 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.109299 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-image-registry-private-configuration\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.109586 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.109569 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-trusted-ca\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.114196 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.114174 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-bound-sa-token\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.114368 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.114353 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qdk6\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-kube-api-access-7qdk6\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.405916 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.405828 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6zc47" event={"ID":"1c2bc53b-e91d-40c4-a177-3724af9c37e9","Type":"ContainerStarted","Data":"86fb88376ffc3a8720e12d96da8accf423511cedc9b290f857d3910af56d60bd"} Jul 28 12:06:47.407552 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.407524 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hk9cz" event={"ID":"804fcdbc-1bb1-4ca3-9e40-e5ab14d8daad","Type":"ContainerStarted","Data":"b07c02dd0fe649c1fbeb25fc79a577abc51610c32fe9cde96fff24fc6cd51cec"} Jul 28 12:06:47.409757 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.409732 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h" event={"ID":"0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7","Type":"ContainerStarted","Data":"8c1da7d6c9dae1f03aad06696a791c6bf733505717a43ea6d50d6a6785836b7e"} Jul 28 12:06:47.411444 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.411136 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" event={"ID":"3dc2810f-7c06-49ad-b0d6-d87a52fd6317","Type":"ContainerStarted","Data":"18dfa375960d5ca627afe29f628619f8425601da34b0cac8c510aa71f18f1f00"} Jul 28 12:06:47.601545 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.601470 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:47.601697 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:47.601579 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 12:06:47.601697 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:47.601601 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-844d6895c9-9l962: secret "image-registry-tls" not found Jul 28 12:06:47.601697 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:47.601665 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls podName:b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0 nodeName:}" failed. No retries permitted until 2026-07-28 12:06:48.601643537 +0000 UTC m=+42.926308796 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls") pod "image-registry-844d6895c9-9l962" (UID: "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0") : secret "image-registry-tls" not found Jul 28 12:06:47.803686 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:47.803652 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dn6dz\" (UID: \"cff40866-ec36-4069-99af-4409e6f10e55\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:06:47.803839 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:47.803807 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 12:06:47.803911 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:47.803894 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls podName:cff40866-ec36-4069-99af-4409e6f10e55 nodeName:}" failed. No retries permitted until 2026-07-28 12:06:49.803872841 +0000 UTC m=+44.128538120 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dn6dz" (UID: "cff40866-ec36-4069-99af-4409e6f10e55") : secret "cluster-monitoring-operator-tls" not found Jul 28 12:06:48.106924 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:48.106792 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:48.106924 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:48.106895 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:48.107239 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:48.106996 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle podName:e2d43bb6-5482-4950-b2a5-b06056f4fa3c nodeName:}" failed. No retries permitted until 2026-07-28 12:06:50.106971062 +0000 UTC m=+44.431636337 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle") pod "router-default-684fd7d5f9-8xm5z" (UID: "e2d43bb6-5482-4950-b2a5-b06056f4fa3c") : configmap references non-existent config key: service-ca.crt Jul 28 12:06:48.107239 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:48.107017 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 12:06:48.107239 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:48.107070 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs podName:e2d43bb6-5482-4950-b2a5-b06056f4fa3c nodeName:}" failed. No retries permitted until 2026-07-28 12:06:50.107055169 +0000 UTC m=+44.431720425 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs") pod "router-default-684fd7d5f9-8xm5z" (UID: "e2d43bb6-5482-4950-b2a5-b06056f4fa3c") : secret "router-metrics-certs-default" not found Jul 28 12:06:48.611202 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:48.611167 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:48.611693 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:48.611375 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 12:06:48.611693 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:48.611394 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-844d6895c9-9l962: secret "image-registry-tls" not found Jul 28 12:06:48.611693 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:48.611461 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls podName:b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0 nodeName:}" failed. No retries permitted until 2026-07-28 12:06:50.611441085 +0000 UTC m=+44.936106341 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls") pod "image-registry-844d6895c9-9l962" (UID: "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0") : secret "image-registry-tls" not found Jul 28 12:06:49.417557 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.417527 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:49.417715 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:49.417677 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 12:06:49.417775 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:49.417753 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls podName:557e6458-36ad-4c2d-bc32-ba9540692a3f nodeName:}" failed. No retries permitted until 2026-07-28 12:06:57.41773106 +0000 UTC m=+51.742396331 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls") pod "dns-default-6k26v" (UID: "557e6458-36ad-4c2d-bc32-ba9540692a3f") : secret "dns-default-metrics-tls" not found Jul 28 12:06:49.586748 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.586661 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lxb9z"] Jul 28 12:06:49.603321 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.603274 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lxb9z"] Jul 28 12:06:49.603452 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.603423 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:49.606364 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.606327 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 28 12:06:49.606569 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.606552 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9qrk5\"" Jul 28 12:06:49.606569 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.606559 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 28 12:06:49.607701 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.607684 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 28 12:06:49.607806 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.607725 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 28 12:06:49.618912 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.618890 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fjst\" (UniqueName: \"kubernetes.io/projected/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-api-access-4fjst\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:49.619222 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.618958 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:49.619222 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.619067 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-data-volume\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:49.619222 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.619112 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-crio-socket\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:49.619222 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.619161 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:49.719574 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.719545 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-data-volume\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:49.719717 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.719584 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-crio-socket\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:49.719717 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.719616 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:49.719717 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:49.719681 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 12:06:49.719878 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:49.719768 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm podName:c9fc92bb-eabd-4601-8e88-61f3eca0d83a nodeName:}" failed. No retries permitted until 2026-07-28 12:06:50.219726313 +0000 UTC m=+44.544391572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lxb9z" (UID: "c9fc92bb-eabd-4601-8e88-61f3eca0d83a") : configmap "kube-rbac-proxy" not found Jul 28 12:06:49.719878 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.719779 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-crio-socket\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:49.719878 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.719786 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fjst\" (UniqueName: \"kubernetes.io/projected/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-api-access-4fjst\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:49.719878 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.719841 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:49.720012 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.719889 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-data-volume\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:49.720012 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:49.719966 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 12:06:49.720012 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:49.720007 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls podName:c9fc92bb-eabd-4601-8e88-61f3eca0d83a nodeName:}" failed. No retries permitted until 2026-07-28 12:06:50.21999257 +0000 UTC m=+44.544657833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lxb9z" (UID: "c9fc92bb-eabd-4601-8e88-61f3eca0d83a") : secret "insights-runtime-extractor-tls" not found Jul 28 12:06:49.729664 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.729637 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fjst\" (UniqueName: \"kubernetes.io/projected/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-api-access-4fjst\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:49.820591 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:49.820570 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dn6dz\" (UID: \"cff40866-ec36-4069-99af-4409e6f10e55\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:06:49.820689 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:49.820677 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 12:06:49.820775 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:49.820720 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls podName:cff40866-ec36-4069-99af-4409e6f10e55 nodeName:}" failed. No retries permitted until 2026-07-28 12:06:53.820708868 +0000 UTC m=+48.145374125 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dn6dz" (UID: "cff40866-ec36-4069-99af-4409e6f10e55") : secret "cluster-monitoring-operator-tls" not found Jul 28 12:06:50.123105 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:50.123073 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:50.123287 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:50.123173 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:50.123287 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:50.123226 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle podName:e2d43bb6-5482-4950-b2a5-b06056f4fa3c nodeName:}" failed. No retries permitted until 2026-07-28 12:06:54.123203576 +0000 UTC m=+48.447868847 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle") pod "router-default-684fd7d5f9-8xm5z" (UID: "e2d43bb6-5482-4950-b2a5-b06056f4fa3c") : configmap references non-existent config key: service-ca.crt Jul 28 12:06:50.123287 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:50.123280 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 12:06:50.123473 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:50.123354 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs podName:e2d43bb6-5482-4950-b2a5-b06056f4fa3c nodeName:}" failed. No retries permitted until 2026-07-28 12:06:54.123338512 +0000 UTC m=+48.448003785 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs") pod "router-default-684fd7d5f9-8xm5z" (UID: "e2d43bb6-5482-4950-b2a5-b06056f4fa3c") : secret "router-metrics-certs-default" not found Jul 28 12:06:50.223688 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:50.223652 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:50.223854 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:50.223798 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:50.223854 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:50.223836 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 12:06:50.223959 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:50.223906 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 12:06:50.223959 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:50.223909 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls podName:c9fc92bb-eabd-4601-8e88-61f3eca0d83a nodeName:}" failed. No retries permitted until 2026-07-28 12:06:51.223889839 +0000 UTC m=+45.548555115 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lxb9z" (UID: "c9fc92bb-eabd-4601-8e88-61f3eca0d83a") : secret "insights-runtime-extractor-tls" not found Jul 28 12:06:50.223959 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:50.223957 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm podName:c9fc92bb-eabd-4601-8e88-61f3eca0d83a nodeName:}" failed. No retries permitted until 2026-07-28 12:06:51.223944086 +0000 UTC m=+45.548609342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lxb9z" (UID: "c9fc92bb-eabd-4601-8e88-61f3eca0d83a") : configmap "kube-rbac-proxy" not found Jul 28 12:06:50.626248 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:50.626205 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:50.626806 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:50.626388 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 12:06:50.626806 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:50.626413 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-844d6895c9-9l962: secret "image-registry-tls" not found Jul 28 12:06:50.626806 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:50.626482 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls podName:b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0 nodeName:}" failed. No retries permitted until 2026-07-28 12:06:54.626461243 +0000 UTC m=+48.951126511 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls") pod "image-registry-844d6895c9-9l962" (UID: "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0") : secret "image-registry-tls" not found Jul 28 12:06:51.229781 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:51.229751 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:51.229896 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:51.229842 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:51.229954 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:51.229891 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 12:06:51.229954 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:51.229917 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 12:06:51.229954 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:51.229953 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls podName:c9fc92bb-eabd-4601-8e88-61f3eca0d83a nodeName:}" failed. No retries permitted until 2026-07-28 12:06:53.229937556 +0000 UTC m=+47.554602812 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lxb9z" (UID: "c9fc92bb-eabd-4601-8e88-61f3eca0d83a") : secret "insights-runtime-extractor-tls" not found Jul 28 12:06:51.230066 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:51.229971 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm podName:c9fc92bb-eabd-4601-8e88-61f3eca0d83a nodeName:}" failed. No retries permitted until 2026-07-28 12:06:53.22996119 +0000 UTC m=+47.554626445 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lxb9z" (UID: "c9fc92bb-eabd-4601-8e88-61f3eca0d83a") : configmap "kube-rbac-proxy" not found Jul 28 12:06:51.343622 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:51.343603 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n97lp_1c56eeda-c493-4055-87eb-1d2fea4a014d/dns-node-resolver/0.log" Jul 28 12:06:52.143987 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:52.143958 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d4d6q_adc764ba-183f-42ff-b0b9-475dcda06640/node-ca/0.log" Jul 28 12:06:52.424859 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:52.424827 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h" event={"ID":"0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7","Type":"ContainerStarted","Data":"0f0c24f1665adb0bc4dea636a0a495a6c8cab63ad0a8fec2c62bb75f4ffebf62"} Jul 28 12:06:52.426480 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:52.426460 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/0.log" Jul 28 12:06:52.426609 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:52.426494 2575 generic.go:356] "Generic (PLEG): container finished" podID="3dc2810f-7c06-49ad-b0d6-d87a52fd6317" containerID="0a3bc0e7d47994d95e62c3cd28b64efe4f93b6637244afa9b0db80dd68eb896c" exitCode=255 Jul 28 12:06:52.426609 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:52.426521 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" event={"ID":"3dc2810f-7c06-49ad-b0d6-d87a52fd6317","Type":"ContainerDied","Data":"0a3bc0e7d47994d95e62c3cd28b64efe4f93b6637244afa9b0db80dd68eb896c"} Jul 28 12:06:52.426760 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:52.426745 2575 scope.go:117] "RemoveContainer" containerID="0a3bc0e7d47994d95e62c3cd28b64efe4f93b6637244afa9b0db80dd68eb896c" Jul 28 12:06:52.428188 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:52.428165 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6zc47" event={"ID":"1c2bc53b-e91d-40c4-a177-3724af9c37e9","Type":"ContainerStarted","Data":"9b4284e5536d5e63f44b9d45383ad64095c90f74c6f4f0f36916077d21ff07e7"} Jul 28 12:06:52.430077 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:52.430059 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hk9cz" event={"ID":"804fcdbc-1bb1-4ca3-9e40-e5ab14d8daad","Type":"ContainerStarted","Data":"7c3bc659bce7d6f987072dc22fa54652d6f94ac0422dd0c4049f41a30108aef0"} Jul 28 12:06:52.443390 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:52.443348 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h" podStartSLOduration=1.7396901169999999 podStartE2EDuration="6.443332245s" totalImagesPullingTime="4.703642128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:06:52.441928694 +0000 UTC m=+46.766593972" watchObservedRunningTime="2026-07-28 12:06:52.443332245 +0000 UTC m=+46.767997524" Jul 28 12:06:52.459450 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:52.459402 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-6zc47" podStartSLOduration=1.8855467639999999 podStartE2EDuration="6.459384501s" totalImagesPullingTime="4.573837737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:06:52.458647349 +0000 UTC m=+46.783312627" watchObservedRunningTime="2026-07-28 12:06:52.459384501 +0000 UTC m=+46.784049782" Jul 28 12:06:52.478729 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:52.478680 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-hk9cz" podStartSLOduration=1.8087174849999998 podStartE2EDuration="6.478662085s" totalImagesPullingTime="4.6699446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:06:52.47719522 +0000 UTC m=+46.801860497" watchObservedRunningTime="2026-07-28 12:06:52.478662085 +0000 UTC m=+46.803327364" Jul 28 12:06:52.943825 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:52.943796 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x4qcv_1974e756-46ac-42da-886f-8353ebd17e41/serve-healthcheck-canary/0.log" Jul 28 12:06:53.249600 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:53.249529 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:53.249941 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:53.249601 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:53.249941 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:53.249654 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 12:06:53.249941 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:53.249721 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm podName:c9fc92bb-eabd-4601-8e88-61f3eca0d83a nodeName:}" failed. No retries permitted until 2026-07-28 12:06:57.24970391 +0000 UTC m=+51.574369186 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lxb9z" (UID: "c9fc92bb-eabd-4601-8e88-61f3eca0d83a") : configmap "kube-rbac-proxy" not found Jul 28 12:06:53.249941 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:53.249768 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 12:06:53.249941 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:53.249835 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls podName:c9fc92bb-eabd-4601-8e88-61f3eca0d83a nodeName:}" failed. No retries permitted until 2026-07-28 12:06:57.249819099 +0000 UTC m=+51.574484370 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lxb9z" (UID: "c9fc92bb-eabd-4601-8e88-61f3eca0d83a") : secret "insights-runtime-extractor-tls" not found Jul 28 12:06:53.435295 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:53.435266 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/1.log" Jul 28 12:06:53.435697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:53.435683 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/0.log" Jul 28 12:06:53.435768 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:53.435722 2575 generic.go:356] "Generic (PLEG): container finished" podID="3dc2810f-7c06-49ad-b0d6-d87a52fd6317" containerID="5eb5127b490392c3c6f12818ccc3b02eb96d25ec1345f599292b0daa5c737a89" exitCode=255 Jul 28 12:06:53.435849 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:53.435825 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" event={"ID":"3dc2810f-7c06-49ad-b0d6-d87a52fd6317","Type":"ContainerDied","Data":"5eb5127b490392c3c6f12818ccc3b02eb96d25ec1345f599292b0daa5c737a89"} Jul 28 12:06:53.435930 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:53.435918 2575 scope.go:117] "RemoveContainer" containerID="0a3bc0e7d47994d95e62c3cd28b64efe4f93b6637244afa9b0db80dd68eb896c" Jul 28 12:06:53.436166 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:53.436147 2575 scope.go:117] "RemoveContainer" containerID="5eb5127b490392c3c6f12818ccc3b02eb96d25ec1345f599292b0daa5c737a89" Jul 28 12:06:53.436607 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:53.436328 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-l4pfh_openshift-console-operator(3dc2810f-7c06-49ad-b0d6-d87a52fd6317)\"" pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" podUID="3dc2810f-7c06-49ad-b0d6-d87a52fd6317" Jul 28 12:06:53.854994 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:53.854968 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dn6dz\" (UID: \"cff40866-ec36-4069-99af-4409e6f10e55\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:06:53.855108 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:53.855091 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 12:06:53.855155 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:53.855148 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls podName:cff40866-ec36-4069-99af-4409e6f10e55 nodeName:}" failed. No retries permitted until 2026-07-28 12:07:01.855134664 +0000 UTC m=+56.179799920 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dn6dz" (UID: "cff40866-ec36-4069-99af-4409e6f10e55") : secret "cluster-monitoring-operator-tls" not found Jul 28 12:06:54.157543 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:54.157483 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:54.157662 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:54.157567 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:06:54.157662 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:54.157627 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 12:06:54.157735 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:54.157669 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle podName:e2d43bb6-5482-4950-b2a5-b06056f4fa3c nodeName:}" failed. No retries permitted until 2026-07-28 12:07:02.157653476 +0000 UTC m=+56.482318734 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle") pod "router-default-684fd7d5f9-8xm5z" (UID: "e2d43bb6-5482-4950-b2a5-b06056f4fa3c") : configmap references non-existent config key: service-ca.crt Jul 28 12:06:54.157735 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:54.157689 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs podName:e2d43bb6-5482-4950-b2a5-b06056f4fa3c nodeName:}" failed. No retries permitted until 2026-07-28 12:07:02.157682789 +0000 UTC m=+56.482348044 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs") pod "router-default-684fd7d5f9-8xm5z" (UID: "e2d43bb6-5482-4950-b2a5-b06056f4fa3c") : secret "router-metrics-certs-default" not found Jul 28 12:06:54.440245 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:54.440219 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/1.log" Jul 28 12:06:54.440585 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:54.440520 2575 scope.go:117] "RemoveContainer" containerID="5eb5127b490392c3c6f12818ccc3b02eb96d25ec1345f599292b0daa5c737a89" Jul 28 12:06:54.440671 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:54.440652 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-l4pfh_openshift-console-operator(3dc2810f-7c06-49ad-b0d6-d87a52fd6317)\"" pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" podUID="3dc2810f-7c06-49ad-b0d6-d87a52fd6317" Jul 28 12:06:54.661277 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:54.661245 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:06:54.661417 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:54.661400 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 12:06:54.661458 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:54.661419 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-844d6895c9-9l962: secret "image-registry-tls" not found Jul 28 12:06:54.661489 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:54.661482 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls podName:b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0 nodeName:}" failed. No retries permitted until 2026-07-28 12:07:02.661464328 +0000 UTC m=+56.986129593 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls") pod "image-registry-844d6895c9-9l962" (UID: "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0") : secret "image-registry-tls" not found Jul 28 12:06:56.637402 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:56.637361 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:06:56.637402 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:56.637412 2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:06:56.637780 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:56.637708 2575 scope.go:117] "RemoveContainer" containerID="5eb5127b490392c3c6f12818ccc3b02eb96d25ec1345f599292b0daa5c737a89" Jul 28 12:06:56.637850 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:56.637831 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-l4pfh_openshift-console-operator(3dc2810f-7c06-49ad-b0d6-d87a52fd6317)\"" pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" podUID="3dc2810f-7c06-49ad-b0d6-d87a52fd6317" Jul 28 12:06:57.281715 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:57.281669 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:57.281917 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:57.281803 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:06:57.281917 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:57.281829 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 12:06:57.281917 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:57.281872 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 12:06:57.281917 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:57.281897 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls podName:c9fc92bb-eabd-4601-8e88-61f3eca0d83a nodeName:}" failed. No retries permitted until 2026-07-28 12:07:05.281875039 +0000 UTC m=+59.606540314 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lxb9z" (UID: "c9fc92bb-eabd-4601-8e88-61f3eca0d83a") : secret "insights-runtime-extractor-tls" not found Jul 28 12:06:57.281917 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:57.281915 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm podName:c9fc92bb-eabd-4601-8e88-61f3eca0d83a nodeName:}" failed. No retries permitted until 2026-07-28 12:07:05.281905187 +0000 UTC m=+59.606570444 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lxb9z" (UID: "c9fc92bb-eabd-4601-8e88-61f3eca0d83a") : configmap "kube-rbac-proxy" not found Jul 28 12:06:57.483864 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:06:57.483818 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:06:57.484012 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:57.483953 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 12:06:57.484012 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:06:57.484006 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls podName:557e6458-36ad-4c2d-bc32-ba9540692a3f nodeName:}" failed. No retries permitted until 2026-07-28 12:07:13.483991137 +0000 UTC m=+67.808656393 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls") pod "dns-default-6k26v" (UID: "557e6458-36ad-4c2d-bc32-ba9540692a3f") : secret "dns-default-metrics-tls" not found Jul 28 12:07:01.915267 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:01.915233 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dn6dz\" (UID: \"cff40866-ec36-4069-99af-4409e6f10e55\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:07:01.915789 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:07:01.915357 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 12:07:01.915789 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:07:01.915418 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls podName:cff40866-ec36-4069-99af-4409e6f10e55 nodeName:}" failed. No retries permitted until 2026-07-28 12:07:17.915403781 +0000 UTC m=+72.240069036 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-dn6dz" (UID: "cff40866-ec36-4069-99af-4409e6f10e55") : secret "cluster-monitoring-operator-tls" not found Jul 28 12:07:02.217344 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:02.217300 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:07:02.217505 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:02.217410 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:07:02.217505 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:07:02.217439 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 12:07:02.217505 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:07:02.217496 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs podName:e2d43bb6-5482-4950-b2a5-b06056f4fa3c nodeName:}" failed. No retries permitted until 2026-07-28 12:07:18.217481037 +0000 UTC m=+72.542146293 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs") pod "router-default-684fd7d5f9-8xm5z" (UID: "e2d43bb6-5482-4950-b2a5-b06056f4fa3c") : secret "router-metrics-certs-default" not found Jul 28 12:07:02.217613 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:07:02.217510 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle podName:e2d43bb6-5482-4950-b2a5-b06056f4fa3c nodeName:}" failed. No retries permitted until 2026-07-28 12:07:18.217503793 +0000 UTC m=+72.542169049 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle") pod "router-default-684fd7d5f9-8xm5z" (UID: "e2d43bb6-5482-4950-b2a5-b06056f4fa3c") : configmap references non-existent config key: service-ca.crt Jul 28 12:07:02.721616 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:02.721582 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls\") pod \"image-registry-844d6895c9-9l962\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:07:02.721779 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:07:02.721710 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 12:07:02.721779 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:07:02.721725 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-844d6895c9-9l962: secret "image-registry-tls" not found Jul 28 12:07:02.721779 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:07:02.721778 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls podName:b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0 nodeName:}" failed. No retries permitted until 2026-07-28 12:07:18.721763835 +0000 UTC m=+73.046429091 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls") pod "image-registry-844d6895c9-9l962" (UID: "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0") : secret "image-registry-tls" not found Jul 28 12:07:05.341553 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:05.341525 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:07:05.341902 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:05.341621 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:07:05.341902 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:07:05.341672 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 12:07:05.341902 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:07:05.341729 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls podName:c9fc92bb-eabd-4601-8e88-61f3eca0d83a nodeName:}" failed. No retries permitted until 2026-07-28 12:07:21.341714919 +0000 UTC m=+75.666380180 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lxb9z" (UID: "c9fc92bb-eabd-4601-8e88-61f3eca0d83a") : secret "insights-runtime-extractor-tls" not found Jul 28 12:07:05.342068 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:05.342052 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:07:06.373431 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:06.373397 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v95xv" Jul 28 12:07:08.204057 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:08.204024 2575 scope.go:117] "RemoveContainer" containerID="5eb5127b490392c3c6f12818ccc3b02eb96d25ec1345f599292b0daa5c737a89" Jul 28 12:07:08.478455 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:08.478379 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/1.log" Jul 28 12:07:08.478612 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:08.478454 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" event={"ID":"3dc2810f-7c06-49ad-b0d6-d87a52fd6317","Type":"ContainerStarted","Data":"b4d0709861f7666c993ef76ccd87c4698850600144e52b769cb3c6a00dc0be1d"} Jul 28 12:07:08.478767 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:08.478749 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:07:08.499210 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:08.499160 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" podStartSLOduration=17.941541674 podStartE2EDuration="22.499145353s" totalImagesPullingTime="4.557603679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:08.498260807 +0000 UTC m=+62.822926084" watchObservedRunningTime="2026-07-28 12:07:08.499145353 +0000 UTC m=+62.823810632" Jul 28 12:07:08.894550 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:08.894485 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-l4pfh" Jul 28 12:07:11.890198 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:11.890162 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs\") pod \"network-metrics-daemon-5f72v\" (UID: \"92daaf51-12ba-4eab-87ac-2087d60aac39\") " pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:07:11.893116 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:11.893094 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 12:07:11.902578 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:11.902557 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92daaf51-12ba-4eab-87ac-2087d60aac39-metrics-certs\") pod \"network-metrics-daemon-5f72v\" (UID: \"92daaf51-12ba-4eab-87ac-2087d60aac39\") " pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:07:11.921512 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:11.921495 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n7r5r\"" Jul 28 12:07:11.929110 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:11.929086 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5f72v" Jul 28 12:07:11.991199 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:11.991173 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret\") pod \"global-pull-secret-syncer-kkbtd\" (UID: \"0f3bc13c-e0e8-44a6-9885-5bcd094112ac\") " pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:07:11.991386 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:11.991257 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nt9n\" (UniqueName: \"kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n\") pod \"network-check-target-fd7d6\" (UID: \"bd0bd704-8198-4017-a6d4-f8222009e54e\") " pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:07:11.993842 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:11.993817 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nt9n\" (UniqueName: \"kubernetes.io/projected/bd0bd704-8198-4017-a6d4-f8222009e54e-kube-api-access-6nt9n\") pod \"network-check-target-fd7d6\" (UID: \"bd0bd704-8198-4017-a6d4-f8222009e54e\") " pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:07:11.994522 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:11.994504 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 12:07:12.003674 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:12.003655 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f3bc13c-e0e8-44a6-9885-5bcd094112ac-original-pull-secret\") pod \"global-pull-secret-syncer-kkbtd\" (UID: \"0f3bc13c-e0e8-44a6-9885-5bcd094112ac\") " pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:07:12.014627 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:12.014606 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkbtd" Jul 28 12:07:12.052366 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:12.052346 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5f72v"] Jul 28 12:07:12.055400 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:12.055361 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92daaf51_12ba_4eab_87ac_2087d60aac39.slice/crio-139e8872da09ba0fa665879e713002436ac1ec6459599767d1e04d3b67a0c981 WatchSource:0}: Error finding container 139e8872da09ba0fa665879e713002436ac1ec6459599767d1e04d3b67a0c981: Status 404 returned error can't find the container with id 139e8872da09ba0fa665879e713002436ac1ec6459599767d1e04d3b67a0c981 Jul 28 12:07:12.163910 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:12.163890 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kkbtd"] Jul 28 12:07:12.164987 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:12.164963 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f3bc13c_e0e8_44a6_9885_5bcd094112ac.slice/crio-31ac4ea49039a6bd8408d3a56c8eefa9eb21bde5386d667fdc6c7ae472b21b31 WatchSource:0}: Error finding container 31ac4ea49039a6bd8408d3a56c8eefa9eb21bde5386d667fdc6c7ae472b21b31: Status 404 returned error can't find the container with id 31ac4ea49039a6bd8408d3a56c8eefa9eb21bde5386d667fdc6c7ae472b21b31 Jul 28 12:07:12.227857 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:12.227832 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qjs5z\"" Jul 28 12:07:12.235362 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:12.235345 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:07:12.382122 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:12.381976 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fd7d6"] Jul 28 12:07:12.384405 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:12.384377 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0bd704_8198_4017_a6d4_f8222009e54e.slice/crio-69a962295abfb674727e9fea1b2afd6f6c44d64e733f09fb19f65e0aaad0e52c WatchSource:0}: Error finding container 69a962295abfb674727e9fea1b2afd6f6c44d64e733f09fb19f65e0aaad0e52c: Status 404 returned error can't find the container with id 69a962295abfb674727e9fea1b2afd6f6c44d64e733f09fb19f65e0aaad0e52c Jul 28 12:07:12.491887 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:12.491860 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fd7d6" event={"ID":"bd0bd704-8198-4017-a6d4-f8222009e54e","Type":"ContainerStarted","Data":"14b4fa7bef89e3714c0cad540f7e504dd0d415416f73ee8813a73457663c5025"} Jul 28 12:07:12.491977 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:12.491946 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fd7d6" event={"ID":"bd0bd704-8198-4017-a6d4-f8222009e54e","Type":"ContainerStarted","Data":"69a962295abfb674727e9fea1b2afd6f6c44d64e733f09fb19f65e0aaad0e52c"} Jul 28 12:07:12.493029 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:12.493003 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5f72v" event={"ID":"92daaf51-12ba-4eab-87ac-2087d60aac39","Type":"ContainerStarted","Data":"139e8872da09ba0fa665879e713002436ac1ec6459599767d1e04d3b67a0c981"} Jul 28 12:07:12.494032 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:12.494006 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kkbtd" event={"ID":"0f3bc13c-e0e8-44a6-9885-5bcd094112ac","Type":"ContainerStarted","Data":"31ac4ea49039a6bd8408d3a56c8eefa9eb21bde5386d667fdc6c7ae472b21b31"} Jul 28 12:07:13.498475 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:13.498446 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:07:13.502788 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:13.502767 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:07:13.505385 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:13.505363 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/557e6458-36ad-4c2d-bc32-ba9540692a3f-metrics-tls\") pod \"dns-default-6k26v\" (UID: \"557e6458-36ad-4c2d-bc32-ba9540692a3f\") " pod="openshift-dns/dns-default-6k26v" Jul 28 12:07:13.516386 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:13.516341 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fd7d6" podStartSLOduration=67.516324337 podStartE2EDuration="1m7.516324337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:06:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:13.516236824 +0000 UTC m=+67.840902102" watchObservedRunningTime="2026-07-28 12:07:13.516324337 +0000 UTC m=+67.840989612" Jul 28 12:07:13.699366 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:13.699337 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4j8p8\"" Jul 28 12:07:13.707072 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:13.707055 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6k26v" Jul 28 12:07:13.847862 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:13.847832 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6k26v"] Jul 28 12:07:13.850601 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:13.850576 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod557e6458_36ad_4c2d_bc32_ba9540692a3f.slice/crio-d25d2534c95cfedef2fc31ff1b03946f7e45692746db35d9a9f38f9b1c0ea00d WatchSource:0}: Error finding container d25d2534c95cfedef2fc31ff1b03946f7e45692746db35d9a9f38f9b1c0ea00d: Status 404 returned error can't find the container with id d25d2534c95cfedef2fc31ff1b03946f7e45692746db35d9a9f38f9b1c0ea00d Jul 28 12:07:14.503640 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.503575 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6k26v" event={"ID":"557e6458-36ad-4c2d-bc32-ba9540692a3f","Type":"ContainerStarted","Data":"d25d2534c95cfedef2fc31ff1b03946f7e45692746db35d9a9f38f9b1c0ea00d"} Jul 28 12:07:14.506311 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.506270 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5f72v" event={"ID":"92daaf51-12ba-4eab-87ac-2087d60aac39","Type":"ContainerStarted","Data":"1252944936edd0151e69d98e56ae0d65a910eef353f40dac20c65ca9b9470a8a"} Jul 28 12:07:14.506456 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.506349 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5f72v" event={"ID":"92daaf51-12ba-4eab-87ac-2087d60aac39","Type":"ContainerStarted","Data":"d6c06973c604fbfd28eeca1530cf2b27804551b70c706a020b7de3c9cad3e7aa"} Jul 28 12:07:14.544518 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.544465 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5f72v" podStartSLOduration=67.173055465 podStartE2EDuration="1m8.544445458s" totalImagesPullingTime="1.371389993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:14.543468625 +0000 UTC m=+68.868133905" watchObservedRunningTime="2026-07-28 12:07:14.544445458 +0000 UTC m=+68.869110738" Jul 28 12:07:14.591536 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.591505 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-844d6895c9-9l962"] Jul 28 12:07:14.591777 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:07:14.591752 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-844d6895c9-9l962" podUID="b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0" Jul 28 12:07:14.670428 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.670394 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-mb9dv"] Jul 28 12:07:14.672918 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.672892 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-86snf"] Jul 28 12:07:14.673068 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.673053 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-mb9dv" Jul 28 12:07:14.675004 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.674984 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-86snf" Jul 28 12:07:14.683599 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.683581 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 28 12:07:14.683702 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.683682 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-bb7rs\"" Jul 28 12:07:14.683781 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.683743 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 28 12:07:14.684502 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.684482 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 28 12:07:14.686643 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.686563 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 28 12:07:14.687084 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.687063 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-692n2\"" Jul 28 12:07:14.712416 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.712392 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjs27\" (UniqueName: \"kubernetes.io/projected/fd8e9c98-3dc1-4d79-934f-82a071887365-kube-api-access-wjs27\") pod \"downloads-7c97dc4555-mb9dv\" (UID: \"fd8e9c98-3dc1-4d79-934f-82a071887365\") " pod="openshift-console/downloads-7c97dc4555-mb9dv" Jul 28 12:07:14.712525 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.712443 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e9c376c1-4429-400e-972c-01b99c92943f-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-86snf\" (UID: \"e9c376c1-4429-400e-972c-01b99c92943f\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-86snf" Jul 28 12:07:14.712591 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.712530 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e9c376c1-4429-400e-972c-01b99c92943f-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-86snf\" (UID: \"e9c376c1-4429-400e-972c-01b99c92943f\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-86snf" Jul 28 12:07:14.727454 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.727427 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-86snf"] Jul 28 12:07:14.728569 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.728544 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-mb9dv"] Jul 28 12:07:14.768692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.768621 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-f4c495456-mq4qb"] Jul 28 12:07:14.771524 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.771496 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.804238 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.804177 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f4c495456-mq4qb"] Jul 28 12:07:14.812996 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.812967 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fccb616f-b682-4251-9bdb-5b80ba21d6a5-ca-trust-extracted\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.813114 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.813008 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fccb616f-b682-4251-9bdb-5b80ba21d6a5-bound-sa-token\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.813114 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.813035 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fccb616f-b682-4251-9bdb-5b80ba21d6a5-registry-certificates\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.813114 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.813091 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fccb616f-b682-4251-9bdb-5b80ba21d6a5-installation-pull-secrets\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.813268 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.813126 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t9dv\" (UniqueName: \"kubernetes.io/projected/fccb616f-b682-4251-9bdb-5b80ba21d6a5-kube-api-access-6t9dv\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.813268 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.813163 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fccb616f-b682-4251-9bdb-5b80ba21d6a5-registry-tls\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.813268 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.813242 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjs27\" (UniqueName: \"kubernetes.io/projected/fd8e9c98-3dc1-4d79-934f-82a071887365-kube-api-access-wjs27\") pod \"downloads-7c97dc4555-mb9dv\" (UID: \"fd8e9c98-3dc1-4d79-934f-82a071887365\") " pod="openshift-console/downloads-7c97dc4555-mb9dv" Jul 28 12:07:14.813428 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.813272 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fccb616f-b682-4251-9bdb-5b80ba21d6a5-image-registry-private-configuration\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.813428 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.813352 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e9c376c1-4429-400e-972c-01b99c92943f-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-86snf\" (UID: \"e9c376c1-4429-400e-972c-01b99c92943f\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-86snf" Jul 28 12:07:14.813530 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.813430 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e9c376c1-4429-400e-972c-01b99c92943f-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-86snf\" (UID: \"e9c376c1-4429-400e-972c-01b99c92943f\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-86snf" Jul 28 12:07:14.813530 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.813480 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fccb616f-b682-4251-9bdb-5b80ba21d6a5-trusted-ca\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.814110 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.814093 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e9c376c1-4429-400e-972c-01b99c92943f-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-86snf\" (UID: \"e9c376c1-4429-400e-972c-01b99c92943f\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-86snf" Jul 28 12:07:14.816390 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.816346 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e9c376c1-4429-400e-972c-01b99c92943f-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-86snf\" (UID: \"e9c376c1-4429-400e-972c-01b99c92943f\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-86snf" Jul 28 12:07:14.834069 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.834017 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjs27\" (UniqueName: \"kubernetes.io/projected/fd8e9c98-3dc1-4d79-934f-82a071887365-kube-api-access-wjs27\") pod \"downloads-7c97dc4555-mb9dv\" (UID: \"fd8e9c98-3dc1-4d79-934f-82a071887365\") " pod="openshift-console/downloads-7c97dc4555-mb9dv" Jul 28 12:07:14.914728 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.914706 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fccb616f-b682-4251-9bdb-5b80ba21d6a5-registry-tls\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.914895 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.914765 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fccb616f-b682-4251-9bdb-5b80ba21d6a5-image-registry-private-configuration\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.914895 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.914842 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fccb616f-b682-4251-9bdb-5b80ba21d6a5-trusted-ca\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.915007 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.914902 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fccb616f-b682-4251-9bdb-5b80ba21d6a5-ca-trust-extracted\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.915007 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.914927 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fccb616f-b682-4251-9bdb-5b80ba21d6a5-bound-sa-token\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.915007 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.914968 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fccb616f-b682-4251-9bdb-5b80ba21d6a5-registry-certificates\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.915205 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.915023 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fccb616f-b682-4251-9bdb-5b80ba21d6a5-installation-pull-secrets\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.915205 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.915052 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t9dv\" (UniqueName: \"kubernetes.io/projected/fccb616f-b682-4251-9bdb-5b80ba21d6a5-kube-api-access-6t9dv\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.916431 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.916353 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fccb616f-b682-4251-9bdb-5b80ba21d6a5-ca-trust-extracted\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.921255 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.917024 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fccb616f-b682-4251-9bdb-5b80ba21d6a5-registry-certificates\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.921255 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.917682 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fccb616f-b682-4251-9bdb-5b80ba21d6a5-trusted-ca\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.921255 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.917764 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fccb616f-b682-4251-9bdb-5b80ba21d6a5-registry-tls\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.923368 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.923284 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fccb616f-b682-4251-9bdb-5b80ba21d6a5-image-registry-private-configuration\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.923694 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.923663 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fccb616f-b682-4251-9bdb-5b80ba21d6a5-installation-pull-secrets\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.927086 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.927047 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fccb616f-b682-4251-9bdb-5b80ba21d6a5-bound-sa-token\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.927462 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.927443 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t9dv\" (UniqueName: \"kubernetes.io/projected/fccb616f-b682-4251-9bdb-5b80ba21d6a5-kube-api-access-6t9dv\") pod \"image-registry-f4c495456-mq4qb\" (UID: \"fccb616f-b682-4251-9bdb-5b80ba21d6a5\") " pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:14.986052 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.986031 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-mb9dv" Jul 28 12:07:14.992730 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:14.992704 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-86snf" Jul 28 12:07:15.085769 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.085703 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8rhtj\"" Jul 28 12:07:15.093276 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.093249 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:15.510334 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.510293 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:07:15.516874 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.516841 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:07:15.623477 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.623443 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-certificates\") pod \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " Jul 28 12:07:15.623651 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.623501 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-trusted-ca\") pod \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " Jul 28 12:07:15.623651 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.623535 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-image-registry-private-configuration\") pod \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " Jul 28 12:07:15.623651 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.623581 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-ca-trust-extracted\") pod \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " Jul 28 12:07:15.623651 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.623607 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-bound-sa-token\") pod \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " Jul 28 12:07:15.623651 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.623634 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-installation-pull-secrets\") pod \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " Jul 28 12:07:15.623904 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.623660 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qdk6\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-kube-api-access-7qdk6\") pod \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\" (UID: \"b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0\") " Jul 28 12:07:15.623904 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.623788 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0" (UID: "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:07:15.624321 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.624010 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0" (UID: "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:07:15.624321 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.624039 2575 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-certificates\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:07:15.624321 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.624261 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0" (UID: "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 12:07:15.626466 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.626399 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0" (UID: "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:07:15.626466 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.626425 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0" (UID: "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 12:07:15.626466 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.626460 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0" (UID: "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:07:15.626685 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.626579 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-kube-api-access-7qdk6" (OuterVolumeSpecName: "kube-api-access-7qdk6") pod "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0" (UID: "b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0"). InnerVolumeSpecName "kube-api-access-7qdk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 12:07:15.724710 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.724683 2575 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-ca-trust-extracted\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:07:15.724710 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.724709 2575 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-bound-sa-token\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:07:15.724893 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.724723 2575 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-installation-pull-secrets\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:07:15.724893 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.724738 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qdk6\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-kube-api-access-7qdk6\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:07:15.724893 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.724750 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-trusted-ca\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:07:15.724893 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:15.724758 2575 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-image-registry-private-configuration\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:07:16.514832 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:16.514746 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-844d6895c9-9l962" Jul 28 12:07:16.567266 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:16.567236 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-844d6895c9-9l962"] Jul 28 12:07:16.569850 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:16.569817 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-844d6895c9-9l962"] Jul 28 12:07:16.631907 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:16.631863 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0-registry-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:07:16.795910 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:16.795877 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-mb9dv"] Jul 28 12:07:16.801731 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:16.801699 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd8e9c98_3dc1_4d79_934f_82a071887365.slice/crio-0bad5dbdaad946461aa059a7ea798d34567390e40588c5224de82004309dc9bc WatchSource:0}: Error finding container 0bad5dbdaad946461aa059a7ea798d34567390e40588c5224de82004309dc9bc: Status 404 returned error can't find the container with id 0bad5dbdaad946461aa059a7ea798d34567390e40588c5224de82004309dc9bc Jul 28 12:07:16.818694 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:16.818670 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f4c495456-mq4qb"] Jul 28 12:07:16.822233 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:16.822203 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfccb616f_b682_4251_9bdb_5b80ba21d6a5.slice/crio-9853c14378f6af827b3ce518245b8a4af53f4408a3e2028b37b7ea81f20c5497 WatchSource:0}: Error finding container 9853c14378f6af827b3ce518245b8a4af53f4408a3e2028b37b7ea81f20c5497: Status 404 returned error can't find the container with id 9853c14378f6af827b3ce518245b8a4af53f4408a3e2028b37b7ea81f20c5497 Jul 28 12:07:16.835927 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:16.833104 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-86snf"] Jul 28 12:07:16.841468 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:16.841444 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9c376c1_4429_400e_972c_01b99c92943f.slice/crio-0be0a251712574cb0578a7672ed8ce118473039e305722c66951be39a9516fdb WatchSource:0}: Error finding container 0be0a251712574cb0578a7672ed8ce118473039e305722c66951be39a9516fdb: Status 404 returned error can't find the container with id 0be0a251712574cb0578a7672ed8ce118473039e305722c66951be39a9516fdb Jul 28 12:07:17.520109 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:17.520073 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kkbtd" event={"ID":"0f3bc13c-e0e8-44a6-9885-5bcd094112ac","Type":"ContainerStarted","Data":"9879ac36581951164956756bcb3880804f689fdc00a0855b8d8ba75222dee505"} Jul 28 12:07:17.521356 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:17.521331 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-86snf" event={"ID":"e9c376c1-4429-400e-972c-01b99c92943f","Type":"ContainerStarted","Data":"0be0a251712574cb0578a7672ed8ce118473039e305722c66951be39a9516fdb"} Jul 28 12:07:17.523173 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:17.523151 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6k26v" event={"ID":"557e6458-36ad-4c2d-bc32-ba9540692a3f","Type":"ContainerStarted","Data":"9951e2698d156789fb3abca36a3385d5c66d98797546ff8b0f47ab4c10fd2a2f"} Jul 28 12:07:17.523298 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:17.523196 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6k26v" event={"ID":"557e6458-36ad-4c2d-bc32-ba9540692a3f","Type":"ContainerStarted","Data":"39de7ba5683a327f32fb0710b3a613ec173c5c5d34a9a06d5a54dcccec47ab60"} Jul 28 12:07:17.523298 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:17.523223 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6k26v" Jul 28 12:07:17.524611 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:17.524585 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f4c495456-mq4qb" event={"ID":"fccb616f-b682-4251-9bdb-5b80ba21d6a5","Type":"ContainerStarted","Data":"7a3a7f9ef0d7a53e83a799808cd3a46c545ba2dcae4dd7a1a16ddf66b0185dba"} Jul 28 12:07:17.524718 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:17.524642 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f4c495456-mq4qb" event={"ID":"fccb616f-b682-4251-9bdb-5b80ba21d6a5","Type":"ContainerStarted","Data":"9853c14378f6af827b3ce518245b8a4af53f4408a3e2028b37b7ea81f20c5497"} Jul 28 12:07:17.524785 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:17.524725 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:17.525742 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:17.525722 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-mb9dv" event={"ID":"fd8e9c98-3dc1-4d79-934f-82a071887365","Type":"ContainerStarted","Data":"0bad5dbdaad946461aa059a7ea798d34567390e40588c5224de82004309dc9bc"} Jul 28 12:07:17.539745 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:17.539687 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kkbtd" podStartSLOduration=67.076098848 podStartE2EDuration="1m11.539674783s" totalImagesPullingTime="4.463575935s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:17.538288069 +0000 UTC m=+71.862953347" watchObservedRunningTime="2026-07-28 12:07:17.539674783 +0000 UTC m=+71.864340052" Jul 28 12:07:17.561148 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:17.561105 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6k26v" podStartSLOduration=33.783293174 podStartE2EDuration="36.56109243s" totalImagesPullingTime="2.777799256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:06:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:17.55946332 +0000 UTC m=+71.884128597" watchObservedRunningTime="2026-07-28 12:07:17.56109243 +0000 UTC m=+71.885757709" Jul 28 12:07:17.582793 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:17.582752 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-f4c495456-mq4qb" podStartSLOduration=3.582735662 podStartE2EDuration="3.582735662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:07:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:17.581335126 +0000 UTC m=+71.906000402" watchObservedRunningTime="2026-07-28 12:07:17.582735662 +0000 UTC m=+71.907400923" Jul 28 12:07:17.949033 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:17.949001 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dn6dz\" (UID: \"cff40866-ec36-4069-99af-4409e6f10e55\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:07:17.951591 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:17.951568 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cff40866-ec36-4069-99af-4409e6f10e55-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-dn6dz\" (UID: \"cff40866-ec36-4069-99af-4409e6f10e55\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:07:18.194213 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:18.194179 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kmjbh\"" Jul 28 12:07:18.201833 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:18.201780 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" Jul 28 12:07:18.207767 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:18.207745 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0" path="/var/lib/kubelet/pods/b23a0ea4-0fd3-4ea8-99ff-649c32bd4ec0/volumes" Jul 28 12:07:18.252954 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:18.252417 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:07:18.252954 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:18.252499 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:07:18.253173 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:18.253152 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-service-ca-bundle\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:07:18.254830 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:18.254808 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2d43bb6-5482-4950-b2a5-b06056f4fa3c-metrics-certs\") pod \"router-default-684fd7d5f9-8xm5z\" (UID: \"e2d43bb6-5482-4950-b2a5-b06056f4fa3c\") " pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:07:18.359587 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:18.359555 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz"] Jul 28 12:07:18.363290 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:18.363265 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcff40866_ec36_4069_99af_4409e6f10e55.slice/crio-f444590a0fb189384a67992cfbc0aad67230de7876e296d2e86e60f7df4db5b8 WatchSource:0}: Error finding container f444590a0fb189384a67992cfbc0aad67230de7876e296d2e86e60f7df4db5b8: Status 404 returned error can't find the container with id f444590a0fb189384a67992cfbc0aad67230de7876e296d2e86e60f7df4db5b8 Jul 28 12:07:18.427859 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:18.427833 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kt62m\"" Jul 28 12:07:18.435548 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:18.435410 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:07:18.534815 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:18.532166 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-86snf" event={"ID":"e9c376c1-4429-400e-972c-01b99c92943f","Type":"ContainerStarted","Data":"b85beebe15cdc3840a72ba9b1ac12dd87359e6f419c50ac98d213bd0ca00812d"} Jul 28 12:07:18.537597 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:18.537508 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" event={"ID":"cff40866-ec36-4069-99af-4409e6f10e55","Type":"ContainerStarted","Data":"f444590a0fb189384a67992cfbc0aad67230de7876e296d2e86e60f7df4db5b8"} Jul 28 12:07:18.552191 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:18.552147 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-86snf" podStartSLOduration=3.170471322 podStartE2EDuration="4.552131542s" totalImagesPullingTime="1.38166022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:07:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:18.551791761 +0000 UTC m=+72.876457040" watchObservedRunningTime="2026-07-28 12:07:18.552131542 +0000 UTC m=+72.876796819" Jul 28 12:07:18.607131 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:18.607082 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-684fd7d5f9-8xm5z"] Jul 28 12:07:18.610457 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:18.610429 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d43bb6_5482_4950_b2a5_b06056f4fa3c.slice/crio-39dc99f3c97f11b4ac774df3cc572f306192409a0da15378b1d8e11c9f7559e3 WatchSource:0}: Error finding container 39dc99f3c97f11b4ac774df3cc572f306192409a0da15378b1d8e11c9f7559e3: Status 404 returned error can't find the container with id 39dc99f3c97f11b4ac774df3cc572f306192409a0da15378b1d8e11c9f7559e3 Jul 28 12:07:19.544538 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:19.544499 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" event={"ID":"e2d43bb6-5482-4950-b2a5-b06056f4fa3c","Type":"ContainerStarted","Data":"8a4850035beb86348634b66ac19e34d6498c49df0771d1950d70428291b1b5a2"} Jul 28 12:07:19.545012 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:19.544561 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" event={"ID":"e2d43bb6-5482-4950-b2a5-b06056f4fa3c","Type":"ContainerStarted","Data":"39dc99f3c97f11b4ac774df3cc572f306192409a0da15378b1d8e11c9f7559e3"} Jul 28 12:07:20.436013 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:20.435974 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:07:20.438921 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:20.438900 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:07:20.462065 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:20.461998 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" podStartSLOduration=34.461983415 podStartE2EDuration="34.461983415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:06:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:19.584689487 +0000 UTC m=+73.909354765" watchObservedRunningTime="2026-07-28 12:07:20.461983415 +0000 UTC m=+74.786648688" Jul 28 12:07:20.548454 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:20.548425 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:07:20.549617 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:20.549599 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-684fd7d5f9-8xm5z" Jul 28 12:07:21.380550 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:21.380515 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:07:21.382718 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:21.382697 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9fc92bb-eabd-4601-8e88-61f3eca0d83a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lxb9z\" (UID: \"c9fc92bb-eabd-4601-8e88-61f3eca0d83a\") " pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:07:21.417510 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:21.417489 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9qrk5\"" Jul 28 12:07:21.425244 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:21.425227 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lxb9z" Jul 28 12:07:21.556552 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:21.556517 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" event={"ID":"cff40866-ec36-4069-99af-4409e6f10e55","Type":"ContainerStarted","Data":"9e36022000cf2fd54a70b07ff386c913f0fac8afea7e4de8282fbb47a67e5364"} Jul 28 12:07:21.575211 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:21.575049 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-dn6dz" podStartSLOduration=33.120590381 podStartE2EDuration="35.575026708s" totalImagesPullingTime="2.454436327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:21.574911065 +0000 UTC m=+75.899576345" watchObservedRunningTime="2026-07-28 12:07:21.575026708 +0000 UTC m=+75.899691986" Jul 28 12:07:21.590010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:21.589978 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lxb9z"] Jul 28 12:07:21.592557 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:21.592532 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9fc92bb_eabd_4601_8e88_61f3eca0d83a.slice/crio-fb10b831fce66d88fe588eb9a21d8ec9b71f14db5d0f83d125f3ed38baadbda9 WatchSource:0}: Error finding container fb10b831fce66d88fe588eb9a21d8ec9b71f14db5d0f83d125f3ed38baadbda9: Status 404 returned error can't find the container with id fb10b831fce66d88fe588eb9a21d8ec9b71f14db5d0f83d125f3ed38baadbda9 Jul 28 12:07:22.564134 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:22.564099 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lxb9z" event={"ID":"c9fc92bb-eabd-4601-8e88-61f3eca0d83a","Type":"ContainerStarted","Data":"fb10b831fce66d88fe588eb9a21d8ec9b71f14db5d0f83d125f3ed38baadbda9"} Jul 28 12:07:25.510097 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.510047 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-78mlh"] Jul 28 12:07:25.514510 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.514485 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" Jul 28 12:07:25.518961 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.518932 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bp7pr\"" Jul 28 12:07:25.519172 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.519154 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 28 12:07:25.519264 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.519243 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 28 12:07:25.519581 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.519528 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 28 12:07:25.525274 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.525220 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-78mlh"] Jul 28 12:07:25.612729 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.612695 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbrhk\" (UniqueName: \"kubernetes.io/projected/4d8498e4-1b04-492e-8370-2897365d6314-kube-api-access-cbrhk\") pod \"prometheus-operator-84d69f794d-78mlh\" (UID: \"4d8498e4-1b04-492e-8370-2897365d6314\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" Jul 28 12:07:25.612895 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.612744 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d8498e4-1b04-492e-8370-2897365d6314-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-78mlh\" (UID: \"4d8498e4-1b04-492e-8370-2897365d6314\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" Jul 28 12:07:25.612895 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.612775 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d8498e4-1b04-492e-8370-2897365d6314-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-78mlh\" (UID: \"4d8498e4-1b04-492e-8370-2897365d6314\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" Jul 28 12:07:25.612895 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.612881 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d8498e4-1b04-492e-8370-2897365d6314-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-78mlh\" (UID: \"4d8498e4-1b04-492e-8370-2897365d6314\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" Jul 28 12:07:25.713599 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.713451 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d8498e4-1b04-492e-8370-2897365d6314-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-78mlh\" (UID: \"4d8498e4-1b04-492e-8370-2897365d6314\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" Jul 28 12:07:25.713599 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.713508 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d8498e4-1b04-492e-8370-2897365d6314-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-78mlh\" (UID: \"4d8498e4-1b04-492e-8370-2897365d6314\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" Jul 28 12:07:25.713599 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.713582 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d8498e4-1b04-492e-8370-2897365d6314-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-78mlh\" (UID: \"4d8498e4-1b04-492e-8370-2897365d6314\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" Jul 28 12:07:25.713877 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.713620 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbrhk\" (UniqueName: \"kubernetes.io/projected/4d8498e4-1b04-492e-8370-2897365d6314-kube-api-access-cbrhk\") pod \"prometheus-operator-84d69f794d-78mlh\" (UID: \"4d8498e4-1b04-492e-8370-2897365d6314\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" Jul 28 12:07:25.714330 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.714284 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d8498e4-1b04-492e-8370-2897365d6314-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-78mlh\" (UID: \"4d8498e4-1b04-492e-8370-2897365d6314\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" Jul 28 12:07:25.717405 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.717382 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4d8498e4-1b04-492e-8370-2897365d6314-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-78mlh\" (UID: \"4d8498e4-1b04-492e-8370-2897365d6314\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" Jul 28 12:07:25.717535 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.717516 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d8498e4-1b04-492e-8370-2897365d6314-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-78mlh\" (UID: \"4d8498e4-1b04-492e-8370-2897365d6314\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" Jul 28 12:07:25.726819 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.726793 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbrhk\" (UniqueName: \"kubernetes.io/projected/4d8498e4-1b04-492e-8370-2897365d6314-kube-api-access-cbrhk\") pod \"prometheus-operator-84d69f794d-78mlh\" (UID: \"4d8498e4-1b04-492e-8370-2897365d6314\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" Jul 28 12:07:25.828359 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:25.828290 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" Jul 28 12:07:27.472779 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:27.472671 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-78mlh"] Jul 28 12:07:27.475966 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:27.475930 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d8498e4_1b04_492e_8370_2897365d6314.slice/crio-2509adc808869e88749e2bbbf3809c5803350d91be417a62481922dfd95a494f WatchSource:0}: Error finding container 2509adc808869e88749e2bbbf3809c5803350d91be417a62481922dfd95a494f: Status 404 returned error can't find the container with id 2509adc808869e88749e2bbbf3809c5803350d91be417a62481922dfd95a494f Jul 28 12:07:27.540288 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:27.540270 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6k26v" Jul 28 12:07:27.584459 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:27.584394 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" event={"ID":"4d8498e4-1b04-492e-8370-2897365d6314","Type":"ContainerStarted","Data":"2509adc808869e88749e2bbbf3809c5803350d91be417a62481922dfd95a494f"} Jul 28 12:07:27.585962 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:27.585934 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lxb9z" event={"ID":"c9fc92bb-eabd-4601-8e88-61f3eca0d83a","Type":"ContainerStarted","Data":"e1784f95544ad94a3ab124ab302a1f107d6693e612fe91814215f8f45a5880c5"} Jul 28 12:07:28.231868 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.231561 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-54f6966796-mjljd"] Jul 28 12:07:28.234478 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.234455 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.239037 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.238236 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 28 12:07:28.239037 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.238524 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 28 12:07:28.239037 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.238705 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 28 12:07:28.239555 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.239538 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 28 12:07:28.239759 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.239538 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 28 12:07:28.239938 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.239859 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kdzdd\"" Jul 28 12:07:28.258316 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.256956 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54f6966796-mjljd"] Jul 28 12:07:28.337486 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.337449 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-config\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.337624 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.337508 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-oauth-config\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.337624 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.337540 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-serving-cert\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.337624 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.337561 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4j9g\" (UniqueName: \"kubernetes.io/projected/0e679f3e-80c4-41f3-9f67-8d037dacddc3-kube-api-access-f4j9g\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.337624 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.337577 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-oauth-serving-cert\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.337843 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.337636 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-service-ca\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.439346 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.438059 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-oauth-config\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.439346 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.438103 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-serving-cert\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.439346 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.438131 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4j9g\" (UniqueName: \"kubernetes.io/projected/0e679f3e-80c4-41f3-9f67-8d037dacddc3-kube-api-access-f4j9g\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.439346 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.438159 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-oauth-serving-cert\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.439346 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.438229 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-service-ca\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.439346 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.438256 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-config\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.439346 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.438965 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-config\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.440331 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.440294 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-oauth-serving-cert\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.440471 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.440448 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-service-ca\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.450456 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.443079 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-oauth-config\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.450456 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.443534 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-serving-cert\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.450456 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.450392 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4j9g\" (UniqueName: \"kubernetes.io/projected/0e679f3e-80c4-41f3-9f67-8d037dacddc3-kube-api-access-f4j9g\") pod \"console-54f6966796-mjljd\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.560483 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.560411 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:28.725401 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:28.725247 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54f6966796-mjljd"] Jul 28 12:07:29.069661 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:29.069611 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e679f3e_80c4_41f3_9f67_8d037dacddc3.slice/crio-e7c7ca650b93e7ba5043922ab11bc4b16a36e5b46ae81ee3dd4654bb2b6cb9ef WatchSource:0}: Error finding container e7c7ca650b93e7ba5043922ab11bc4b16a36e5b46ae81ee3dd4654bb2b6cb9ef: Status 404 returned error can't find the container with id e7c7ca650b93e7ba5043922ab11bc4b16a36e5b46ae81ee3dd4654bb2b6cb9ef Jul 28 12:07:29.597435 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:29.597390 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f6966796-mjljd" event={"ID":"0e679f3e-80c4-41f3-9f67-8d037dacddc3","Type":"ContainerStarted","Data":"e7c7ca650b93e7ba5043922ab11bc4b16a36e5b46ae81ee3dd4654bb2b6cb9ef"} Jul 28 12:07:31.608243 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:31.608157 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" event={"ID":"4d8498e4-1b04-492e-8370-2897365d6314","Type":"ContainerStarted","Data":"04bdfffcc47b795de5eadfcc767a6ccc5106da163531bb9b7c19209a3e5b115f"} Jul 28 12:07:38.541586 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:38.541559 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-f4c495456-mq4qb" Jul 28 12:07:40.647225 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:40.647164 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f6966796-mjljd" event={"ID":"0e679f3e-80c4-41f3-9f67-8d037dacddc3","Type":"ContainerStarted","Data":"fd0d603ae4f2efb5a4eb52668296a4039d63df7f9d978d747fd760429a65d628"} Jul 28 12:07:40.650091 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:40.650020 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-mb9dv" event={"ID":"fd8e9c98-3dc1-4d79-934f-82a071887365","Type":"ContainerStarted","Data":"0c4f9538ad2887aff7027b350b4ff46860ba49511e5fcc149e8ed7c263ad4f2a"} Jul 28 12:07:40.650390 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:40.650351 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-mb9dv" Jul 28 12:07:40.653560 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:40.653435 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" event={"ID":"4d8498e4-1b04-492e-8370-2897365d6314","Type":"ContainerStarted","Data":"71a0a8048fb34af1f3ef587fc2e7a35d53afff8725ff9d1e4e6d54d4062e23a0"} Jul 28 12:07:40.655653 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:40.655632 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lxb9z" event={"ID":"c9fc92bb-eabd-4601-8e88-61f3eca0d83a","Type":"ContainerStarted","Data":"4fd0bcab63942b586c4ae89ccc47ee55cb90c320b24d3325399903e26f38050f"} Jul 28 12:07:40.665999 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:40.665972 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-mb9dv" Jul 28 12:07:40.673410 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:40.673359 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-54f6966796-mjljd" podStartSLOduration=1.703276099 podStartE2EDuration="12.67334168s" totalImagesPullingTime="10.970065581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:07:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:40.670148769 +0000 UTC m=+94.994814059" watchObservedRunningTime="2026-07-28 12:07:40.67334168 +0000 UTC m=+94.998006936" Jul 28 12:07:40.705651 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:40.705587 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-mb9dv" podStartSLOduration=3.512763422 podStartE2EDuration="26.705566136s" totalImagesPullingTime="23.192802714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:07:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:40.704980075 +0000 UTC m=+95.029645368" watchObservedRunningTime="2026-07-28 12:07:40.705566136 +0000 UTC m=+95.030231414" Jul 28 12:07:40.744733 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:40.744445 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-78mlh" podStartSLOduration=14.120869922 podStartE2EDuration="15.744421219s" totalImagesPullingTime="1.623551297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:40.743488501 +0000 UTC m=+95.068153781" watchObservedRunningTime="2026-07-28 12:07:40.744421219 +0000 UTC m=+95.069086499" Jul 28 12:07:42.940035 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:42.939997 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rphs9"] Jul 28 12:07:43.028927 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.028891 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58"] Jul 28 12:07:43.029090 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.029033 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.032768 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.032338 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 28 12:07:43.032768 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.032433 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 28 12:07:43.032768 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.032581 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 28 12:07:43.032768 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.032611 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7vm6j\"" Jul 28 12:07:43.065514 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.065456 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58"] Jul 28 12:07:43.065619 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.065599 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.066221 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.066185 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.066366 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.066254 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-975kz\" (UniqueName: \"kubernetes.io/projected/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-kube-api-access-975kz\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.066366 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.066280 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.066486 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.066375 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.066486 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.066415 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.066486 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.066443 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.069267 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.068815 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 28 12:07:43.069267 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.068878 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 28 12:07:43.069267 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.068947 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-mdb69\"" Jul 28 12:07:43.069267 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.069100 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 28 12:07:43.167138 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.167105 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftw76\" (UniqueName: \"kubernetes.io/projected/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-kube-api-access-ftw76\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.167692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.167153 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-root\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.167692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.167187 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-975kz\" (UniqueName: \"kubernetes.io/projected/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-kube-api-access-975kz\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.167692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.167220 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.167692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.167261 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-node-exporter-tls\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.167692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.167283 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.167692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.167327 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.167692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.167380 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.167692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.167459 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.167692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.167501 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-node-exporter-wtmp\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.167692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.167535 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-node-exporter-accelerators-collector-config\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.167692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.167571 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-metrics-client-ca\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.167692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.167608 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-sys\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.167692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.167694 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.169412 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.167747 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-node-exporter-textfile\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.169412 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.167805 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.169412 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.168194 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.169412 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.168241 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.170633 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.170611 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.170633 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.170659 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.178836 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.178762 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-975kz\" (UniqueName: \"kubernetes.io/projected/cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6-kube-api-access-975kz\") pod \"kube-state-metrics-56d9f944b4-hgt58\" (UID: \"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:43.269461 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.268411 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-node-exporter-tls\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.269461 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.268447 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.269461 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.268481 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-node-exporter-wtmp\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.269461 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.268514 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-node-exporter-accelerators-collector-config\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.269461 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.268553 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-metrics-client-ca\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.269461 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.268587 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-sys\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.269461 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.268637 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-node-exporter-textfile\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.269461 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.268701 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftw76\" (UniqueName: \"kubernetes.io/projected/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-kube-api-access-ftw76\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.269461 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.268731 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-root\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.269461 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.268772 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-sys\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.269461 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.268795 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-root\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.269461 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.268960 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-node-exporter-wtmp\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.269461 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.269207 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-node-exporter-textfile\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.269461 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.269355 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-node-exporter-accelerators-collector-config\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.269461 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.269455 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-metrics-client-ca\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.272654 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.272609 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-node-exporter-tls\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.272800 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.272777 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.278850 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.278828 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftw76\" (UniqueName: \"kubernetes.io/projected/9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf-kube-api-access-ftw76\") pod \"node-exporter-rphs9\" (UID: \"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf\") " pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.343105 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.342904 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rphs9" Jul 28 12:07:43.378284 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:43.378216 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" Jul 28 12:07:44.472573 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:44.472525 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9529fbb6_d1b3_4de4_8ff0_ec3ddef8bcaf.slice/crio-fad3da140cdc7357ad3d294b078711fb23c9f9c3df4783530174a2c6a81e14c6 WatchSource:0}: Error finding container fad3da140cdc7357ad3d294b078711fb23c9f9c3df4783530174a2c6a81e14c6: Status 404 returned error can't find the container with id fad3da140cdc7357ad3d294b078711fb23c9f9c3df4783530174a2c6a81e14c6 Jul 28 12:07:44.509605 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:44.508779 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fd7d6" Jul 28 12:07:44.632058 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:44.631507 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58"] Jul 28 12:07:44.635556 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:44.635511 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb2fac3a_f937_4bb5_98a1_4a7341a2dfd6.slice/crio-47fe73c199d86ee3dc6b5032ec2d8abf2183925bd036406efa0879de6dd295ea WatchSource:0}: Error finding container 47fe73c199d86ee3dc6b5032ec2d8abf2183925bd036406efa0879de6dd295ea: Status 404 returned error can't find the container with id 47fe73c199d86ee3dc6b5032ec2d8abf2183925bd036406efa0879de6dd295ea Jul 28 12:07:44.676499 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:44.676467 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rphs9" event={"ID":"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf","Type":"ContainerStarted","Data":"fad3da140cdc7357ad3d294b078711fb23c9f9c3df4783530174a2c6a81e14c6"} Jul 28 12:07:44.677951 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:44.677870 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" event={"ID":"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6","Type":"ContainerStarted","Data":"47fe73c199d86ee3dc6b5032ec2d8abf2183925bd036406efa0879de6dd295ea"} Jul 28 12:07:45.690390 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:45.690286 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lxb9z" event={"ID":"c9fc92bb-eabd-4601-8e88-61f3eca0d83a","Type":"ContainerStarted","Data":"20713f4c08e80cc4d8c3ba4a9811aecbbadb7ff1d58e80207b68ec7880824d98"} Jul 28 12:07:45.723928 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:45.723876 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lxb9z" podStartSLOduration=33.640611005 podStartE2EDuration="56.723852939s" totalImagesPullingTime="23.083241934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:06:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:45.721197356 +0000 UTC m=+100.045862637" watchObservedRunningTime="2026-07-28 12:07:45.723852939 +0000 UTC m=+100.048518218" Jul 28 12:07:47.704510 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.704467 2575 generic.go:356] "Generic (PLEG): container finished" podID="9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf" containerID="0dff3aa77c72adab28683e127e3c8b4fe12e87a90be7b4461c858af865754a08" exitCode=0 Jul 28 12:07:47.704944 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.704565 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rphs9" event={"ID":"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf","Type":"ContainerDied","Data":"0dff3aa77c72adab28683e127e3c8b4fe12e87a90be7b4461c858af865754a08"} Jul 28 12:07:47.708691 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.708429 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" event={"ID":"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6","Type":"ContainerStarted","Data":"9f4f1899b49e50cf2c970f4e5a43b6cc87f10e8b6b5b67f7251bac88bf7f2ff6"} Jul 28 12:07:47.708691 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.708510 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" event={"ID":"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6","Type":"ContainerStarted","Data":"d4b4b5a5549d11557e212e61f67fa52f805123f3fdccb85936e30b5e7acd6a7e"} Jul 28 12:07:47.708691 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.708541 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" event={"ID":"cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6","Type":"ContainerStarted","Data":"75889c4229a20d58ed2a0b1fe0818d91382c64ac9550a05d03bd8a9361b50e70"} Jul 28 12:07:47.731395 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.731275 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-swwfb"] Jul 28 12:07:47.758710 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.758683 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54f6966796-mjljd"] Jul 28 12:07:47.758817 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.758738 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-swwfb"] Jul 28 12:07:47.758926 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.758905 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-swwfb" Jul 28 12:07:47.763428 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.763340 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 28 12:07:47.763719 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.763681 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2nskz\"" Jul 28 12:07:47.780569 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.780504 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-hgt58" podStartSLOduration=3.7039102550000003 podStartE2EDuration="5.780434494s" totalImagesPullingTime="2.076524239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:47.778769769 +0000 UTC m=+102.103435050" watchObservedRunningTime="2026-07-28 12:07:47.780434494 +0000 UTC m=+102.105099777" Jul 28 12:07:47.787132 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.787101 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-97cf9747f-99xkg"] Jul 28 12:07:47.807216 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.807191 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxl69\" (UniqueName: \"kubernetes.io/projected/ff2e5d5b-91bb-4e9a-af0b-604bc8a96b1e-kube-api-access-jxl69\") pod \"monitoring-plugin-68b7fc9574-swwfb\" (UID: \"ff2e5d5b-91bb-4e9a-af0b-604bc8a96b1e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-swwfb" Jul 28 12:07:47.807344 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.807264 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ff2e5d5b-91bb-4e9a-af0b-604bc8a96b1e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-swwfb\" (UID: \"ff2e5d5b-91bb-4e9a-af0b-604bc8a96b1e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-swwfb" Jul 28 12:07:47.810183 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.810164 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:47.813201 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.813065 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-97cf9747f-99xkg"] Jul 28 12:07:47.824571 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.824553 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 28 12:07:47.909596 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.909565 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ff2e5d5b-91bb-4e9a-af0b-604bc8a96b1e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-swwfb\" (UID: \"ff2e5d5b-91bb-4e9a-af0b-604bc8a96b1e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-swwfb" Jul 28 12:07:47.909715 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.909613 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c78be9ee-ce1e-49e3-b532-470765378076-console-serving-cert\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:47.909715 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.909637 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-service-ca\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:47.909715 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.909665 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-oauth-serving-cert\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:47.909715 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.909707 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-console-config\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:47.909919 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.909740 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd59h\" (UniqueName: \"kubernetes.io/projected/c78be9ee-ce1e-49e3-b532-470765378076-kube-api-access-sd59h\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:47.909919 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.909794 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-trusted-ca-bundle\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:47.909919 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.909846 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c78be9ee-ce1e-49e3-b532-470765378076-console-oauth-config\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:47.909919 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.909876 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxl69\" (UniqueName: \"kubernetes.io/projected/ff2e5d5b-91bb-4e9a-af0b-604bc8a96b1e-kube-api-access-jxl69\") pod \"monitoring-plugin-68b7fc9574-swwfb\" (UID: \"ff2e5d5b-91bb-4e9a-af0b-604bc8a96b1e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-swwfb" Jul 28 12:07:47.912386 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.912365 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ff2e5d5b-91bb-4e9a-af0b-604bc8a96b1e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-swwfb\" (UID: \"ff2e5d5b-91bb-4e9a-af0b-604bc8a96b1e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-swwfb" Jul 28 12:07:47.925572 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:47.925553 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxl69\" (UniqueName: \"kubernetes.io/projected/ff2e5d5b-91bb-4e9a-af0b-604bc8a96b1e-kube-api-access-jxl69\") pod \"monitoring-plugin-68b7fc9574-swwfb\" (UID: \"ff2e5d5b-91bb-4e9a-af0b-604bc8a96b1e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-swwfb" Jul 28 12:07:48.010416 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.010393 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-oauth-serving-cert\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:48.010537 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.010447 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-console-config\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:48.010537 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.010477 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd59h\" (UniqueName: \"kubernetes.io/projected/c78be9ee-ce1e-49e3-b532-470765378076-kube-api-access-sd59h\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:48.010537 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.010525 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-trusted-ca-bundle\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:48.010697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.010570 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c78be9ee-ce1e-49e3-b532-470765378076-console-oauth-config\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:48.010697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.010615 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c78be9ee-ce1e-49e3-b532-470765378076-console-serving-cert\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:48.010697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.010639 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-service-ca\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:48.011221 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.011191 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-console-config\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:48.011330 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.011234 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-oauth-serving-cert\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:48.011330 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.011264 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-service-ca\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:48.012153 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.012082 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-trusted-ca-bundle\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:48.014051 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.013997 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c78be9ee-ce1e-49e3-b532-470765378076-console-oauth-config\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:48.014944 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.014923 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c78be9ee-ce1e-49e3-b532-470765378076-console-serving-cert\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:48.032380 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.032353 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd59h\" (UniqueName: \"kubernetes.io/projected/c78be9ee-ce1e-49e3-b532-470765378076-kube-api-access-sd59h\") pod \"console-97cf9747f-99xkg\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:48.094264 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.094237 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-swwfb" Jul 28 12:07:48.123743 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.123717 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:48.159000 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.158979 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc"] Jul 28 12:07:48.188293 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.186546 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc"] Jul 28 12:07:48.188293 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.186689 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.193040 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.193006 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 28 12:07:48.193153 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.193060 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 28 12:07:48.193219 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.193006 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 28 12:07:48.193367 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.193294 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 28 12:07:48.193462 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.193400 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-mll5l\"" Jul 28 12:07:48.193707 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.193623 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 28 12:07:48.202452 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.202275 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 28 12:07:48.215469 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.214570 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/facf572b-b208-4a77-b8c7-f6a229807623-secret-telemeter-client\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.215469 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.214608 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/facf572b-b208-4a77-b8c7-f6a229807623-telemeter-client-tls\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.215469 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.214637 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/facf572b-b208-4a77-b8c7-f6a229807623-metrics-client-ca\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.215469 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.214694 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/facf572b-b208-4a77-b8c7-f6a229807623-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.215469 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.214735 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp74t\" (UniqueName: \"kubernetes.io/projected/facf572b-b208-4a77-b8c7-f6a229807623-kube-api-access-sp74t\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.215469 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.214763 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/facf572b-b208-4a77-b8c7-f6a229807623-federate-client-tls\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.215469 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.214820 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/facf572b-b208-4a77-b8c7-f6a229807623-serving-certs-ca-bundle\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.215469 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.214853 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/facf572b-b208-4a77-b8c7-f6a229807623-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.312946 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.312882 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-swwfb"] Jul 28 12:07:48.315988 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.315957 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp74t\" (UniqueName: \"kubernetes.io/projected/facf572b-b208-4a77-b8c7-f6a229807623-kube-api-access-sp74t\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.316095 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.316004 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/facf572b-b208-4a77-b8c7-f6a229807623-federate-client-tls\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.316095 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.316064 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/facf572b-b208-4a77-b8c7-f6a229807623-serving-certs-ca-bundle\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.316192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.316098 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/facf572b-b208-4a77-b8c7-f6a229807623-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.316192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.316173 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/facf572b-b208-4a77-b8c7-f6a229807623-secret-telemeter-client\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.316284 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.316198 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/facf572b-b208-4a77-b8c7-f6a229807623-telemeter-client-tls\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.316284 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.316227 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/facf572b-b208-4a77-b8c7-f6a229807623-metrics-client-ca\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.316284 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.316266 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/facf572b-b208-4a77-b8c7-f6a229807623-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.318258 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.317602 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/facf572b-b208-4a77-b8c7-f6a229807623-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.320471 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.318784 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/facf572b-b208-4a77-b8c7-f6a229807623-serving-certs-ca-bundle\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.320471 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.319554 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/facf572b-b208-4a77-b8c7-f6a229807623-metrics-client-ca\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.321542 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.321495 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/facf572b-b208-4a77-b8c7-f6a229807623-telemeter-client-tls\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.323001 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.322123 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/facf572b-b208-4a77-b8c7-f6a229807623-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.323421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.323397 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/facf572b-b208-4a77-b8c7-f6a229807623-federate-client-tls\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.324520 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.324501 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/facf572b-b208-4a77-b8c7-f6a229807623-secret-telemeter-client\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.327034 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.327012 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp74t\" (UniqueName: \"kubernetes.io/projected/facf572b-b208-4a77-b8c7-f6a229807623-kube-api-access-sp74t\") pod \"telemeter-client-6f5b5f7cfc-zmmnc\" (UID: \"facf572b-b208-4a77-b8c7-f6a229807623\") " pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.337098 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.337071 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-97cf9747f-99xkg"] Jul 28 12:07:48.341802 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:48.337688 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc78be9ee_ce1e_49e3_b532_470765378076.slice/crio-b945d75d8904cdf633265a4cb72ade24148aff6b77abfe4568c5442a9852b780 WatchSource:0}: Error finding container b945d75d8904cdf633265a4cb72ade24148aff6b77abfe4568c5442a9852b780: Status 404 returned error can't find the container with id b945d75d8904cdf633265a4cb72ade24148aff6b77abfe4568c5442a9852b780 Jul 28 12:07:48.505141 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.505078 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" Jul 28 12:07:48.561646 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.561610 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:07:48.680191 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.680164 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc"] Jul 28 12:07:48.682613 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:48.682571 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfacf572b_b208_4a77_b8c7_f6a229807623.slice/crio-55b9cdb2c6d44441da7be581c5cde901ae3cf122e10182f83129faa9fa4397c1 WatchSource:0}: Error finding container 55b9cdb2c6d44441da7be581c5cde901ae3cf122e10182f83129faa9fa4397c1: Status 404 returned error can't find the container with id 55b9cdb2c6d44441da7be581c5cde901ae3cf122e10182f83129faa9fa4397c1 Jul 28 12:07:48.715024 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.714992 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" event={"ID":"facf572b-b208-4a77-b8c7-f6a229807623","Type":"ContainerStarted","Data":"55b9cdb2c6d44441da7be581c5cde901ae3cf122e10182f83129faa9fa4397c1"} Jul 28 12:07:48.717641 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.717616 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rphs9" event={"ID":"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf","Type":"ContainerStarted","Data":"b602e095cd15a3ce5a6eba8dccef0ef19bf05fdbb2bc16305420fee31e2c97c7"} Jul 28 12:07:48.717740 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.717662 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rphs9" event={"ID":"9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf","Type":"ContainerStarted","Data":"d3029848613b2919d95079768646d4be312377584eb7f4a77395349719ba433d"} Jul 28 12:07:48.718954 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.718931 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-swwfb" event={"ID":"ff2e5d5b-91bb-4e9a-af0b-604bc8a96b1e","Type":"ContainerStarted","Data":"583c6a34ec671e14f5eae169dab5eef1168e4c563d9a760d591cbbda72433edf"} Jul 28 12:07:48.720707 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.720682 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-97cf9747f-99xkg" event={"ID":"c78be9ee-ce1e-49e3-b532-470765378076","Type":"ContainerStarted","Data":"38f2a40f4164b090a8c1e8dd09a97bdb375046ebf964df17f18d671867e76169"} Jul 28 12:07:48.720859 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.720836 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-97cf9747f-99xkg" event={"ID":"c78be9ee-ce1e-49e3-b532-470765378076","Type":"ContainerStarted","Data":"b945d75d8904cdf633265a4cb72ade24148aff6b77abfe4568c5442a9852b780"} Jul 28 12:07:48.740414 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.740369 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rphs9" podStartSLOduration=4.389380714 podStartE2EDuration="6.740351445s" totalImagesPullingTime="2.22975409s" totalInitContainerRuntime="121.216641ms" isStatefulPod=true podCreationTimestamp="2026-07-28 12:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:48.738954083 +0000 UTC m=+103.063619362" watchObservedRunningTime="2026-07-28 12:07:48.740351445 +0000 UTC m=+103.065016723" Jul 28 12:07:48.758632 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:48.758560 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-97cf9747f-99xkg" podStartSLOduration=1.758545786 podStartE2EDuration="1.758545786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:07:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:48.756913342 +0000 UTC m=+103.081578646" watchObservedRunningTime="2026-07-28 12:07:48.758545786 +0000 UTC m=+103.083211063" Jul 28 12:07:49.228221 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.228187 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 12:07:49.250990 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.250929 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.254645 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.254089 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 28 12:07:49.254645 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.254490 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4wlqd\"" Jul 28 12:07:49.255693 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.255673 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 28 12:07:49.255803 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.255703 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 28 12:07:49.255803 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.255775 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 28 12:07:49.255915 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.255815 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 28 12:07:49.255915 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.255906 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 28 12:07:49.256019 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.255969 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 28 12:07:49.259230 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.259073 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 28 12:07:49.261455 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.260033 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 28 12:07:49.261455 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.260179 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 28 12:07:49.261455 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.260441 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 28 12:07:49.261455 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.260680 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 28 12:07:49.262286 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.262178 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 28 12:07:49.262776 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.262643 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-82d0ohm6vu5ei\"" Jul 28 12:07:49.264470 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.264134 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 28 12:07:49.265842 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.265788 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 12:07:49.273759 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.273694 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 28 12:07:49.328180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327357 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28bd0d01-e07e-44c0-be06-886a156d2a6d-config-out\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.328180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327404 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc8gv\" (UniqueName: \"kubernetes.io/projected/28bd0d01-e07e-44c0-be06-886a156d2a6d-kube-api-access-lc8gv\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.328180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327436 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.328180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327460 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.328180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327484 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.328180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327515 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-config\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.328180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327540 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.328180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327567 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.328180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327607 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.328180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327633 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.328180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327656 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.328180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327701 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.328180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327727 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.328180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327750 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.328180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327777 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.328180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327801 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.329072 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327858 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-web-config\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.329072 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327887 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28bd0d01-e07e-44c0-be06-886a156d2a6d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.329072 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327910 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.329072 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.327959 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.428990 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.428944 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.429186 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.428993 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.429186 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429020 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.429186 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429045 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.429186 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429065 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.429186 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429156 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-web-config\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.429538 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429197 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28bd0d01-e07e-44c0-be06-886a156d2a6d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.429538 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429222 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.429538 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429271 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.429538 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429345 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28bd0d01-e07e-44c0-be06-886a156d2a6d-config-out\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.429538 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429368 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc8gv\" (UniqueName: \"kubernetes.io/projected/28bd0d01-e07e-44c0-be06-886a156d2a6d-kube-api-access-lc8gv\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.429538 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429397 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.429538 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429422 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.429538 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429448 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.429538 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429505 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-config\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.429538 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429532 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.431706 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429558 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.431706 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429609 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.431706 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429642 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.431706 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.429666 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.431706 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.430151 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.431706 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.430405 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.431706 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.430645 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.431706 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.430976 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.433407 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.432598 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.433407 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.433037 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.433857 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.433837 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.435333 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.434549 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.435333 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.435165 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.435333 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.435274 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.435551 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.435357 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28bd0d01-e07e-44c0-be06-886a156d2a6d-config-out\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.436448 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.436404 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-web-config\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.436893 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.436853 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28bd0d01-e07e-44c0-be06-886a156d2a6d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.437017 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.436994 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.437333 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.437281 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.437675 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.437544 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.437992 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.437965 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-config\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.438071 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.438055 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.439894 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.439874 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.447479 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.446790 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc8gv\" (UniqueName: \"kubernetes.io/projected/28bd0d01-e07e-44c0-be06-886a156d2a6d-kube-api-access-lc8gv\") pod \"prometheus-k8s-0\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.576149 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.576070 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:07:49.822743 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:07:49.822713 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28bd0d01_e07e_44c0_be06_886a156d2a6d.slice/crio-f2df582d80c1d3d0e119e1412d1578f07a068a444cf0c9c62fea27bb18f30c3e WatchSource:0}: Error finding container f2df582d80c1d3d0e119e1412d1578f07a068a444cf0c9c62fea27bb18f30c3e: Status 404 returned error can't find the container with id f2df582d80c1d3d0e119e1412d1578f07a068a444cf0c9c62fea27bb18f30c3e Jul 28 12:07:49.823819 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:49.823797 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 12:07:50.734710 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:50.734674 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bd0d01-e07e-44c0-be06-886a156d2a6d","Type":"ContainerStarted","Data":"f2df582d80c1d3d0e119e1412d1578f07a068a444cf0c9c62fea27bb18f30c3e"} Jul 28 12:07:51.740480 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:51.740442 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-swwfb" event={"ID":"ff2e5d5b-91bb-4e9a-af0b-604bc8a96b1e","Type":"ContainerStarted","Data":"ba169393e48f99e4c1745d9385334eba5bef38799ef50ee40739c9cffe568efa"} Jul 28 12:07:51.741103 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:51.740635 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-swwfb" Jul 28 12:07:51.745482 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:51.745460 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-swwfb" Jul 28 12:07:51.760929 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:51.760878 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-swwfb" podStartSLOduration=2.451018494 podStartE2EDuration="4.760862381s" totalImagesPullingTime="2.309843887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:07:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:51.759565474 +0000 UTC m=+106.084230751" watchObservedRunningTime="2026-07-28 12:07:51.760862381 +0000 UTC m=+106.085527659" Jul 28 12:07:52.746470 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:52.746448 2575 generic.go:356] "Generic (PLEG): container finished" podID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerID="9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f" exitCode=0 Jul 28 12:07:52.746863 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:52.746536 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bd0d01-e07e-44c0-be06-886a156d2a6d","Type":"ContainerDied","Data":"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f"} Jul 28 12:07:53.752606 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:53.752568 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" event={"ID":"facf572b-b208-4a77-b8c7-f6a229807623","Type":"ContainerStarted","Data":"50f7b93083efdb65a3a33e3dcb80a02b47c3d341a53fc345d217f3557214e54e"} Jul 28 12:07:53.753176 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:53.752646 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" event={"ID":"facf572b-b208-4a77-b8c7-f6a229807623","Type":"ContainerStarted","Data":"b5d59f1883e6cd8f70574976abe038ed5248814bf3b444782450388040fd824d"} Jul 28 12:07:53.753176 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:53.752672 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" event={"ID":"facf572b-b208-4a77-b8c7-f6a229807623","Type":"ContainerStarted","Data":"2990f126332cba596986d9f68de626a3cc1bbc24dae1c446eb512ed871c30d32"} Jul 28 12:07:53.779271 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:53.779221 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6f5b5f7cfc-zmmnc" podStartSLOduration=1.785042154 podStartE2EDuration="5.779204645s" totalImagesPullingTime="3.994162491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:07:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:53.777947035 +0000 UTC m=+108.102612313" watchObservedRunningTime="2026-07-28 12:07:53.779204645 +0000 UTC m=+108.103869924" Jul 28 12:07:54.934672 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:54.934637 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-97cf9747f-99xkg"] Jul 28 12:07:57.775424 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:57.775381 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bd0d01-e07e-44c0-be06-886a156d2a6d","Type":"ContainerStarted","Data":"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657"} Jul 28 12:07:57.775863 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:57.775458 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bd0d01-e07e-44c0-be06-886a156d2a6d","Type":"ContainerStarted","Data":"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce"} Jul 28 12:07:58.124419 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:58.124334 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:07:59.787902 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:59.787867 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bd0d01-e07e-44c0-be06-886a156d2a6d","Type":"ContainerStarted","Data":"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276"} Jul 28 12:07:59.788252 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:59.787961 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bd0d01-e07e-44c0-be06-886a156d2a6d","Type":"ContainerStarted","Data":"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1"} Jul 28 12:07:59.788252 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:59.787989 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bd0d01-e07e-44c0-be06-886a156d2a6d","Type":"ContainerStarted","Data":"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc"} Jul 28 12:07:59.788252 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:59.788013 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bd0d01-e07e-44c0-be06-886a156d2a6d","Type":"ContainerStarted","Data":"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d"} Jul 28 12:07:59.821213 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:07:59.820442 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.961160915 podStartE2EDuration="10.820422539s" totalImagesPullingTime="8.839769042s" totalInitContainerRuntime="19.492582ms" isStatefulPod=false podCreationTimestamp="2026-07-28 12:07:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:07:59.817639628 +0000 UTC m=+114.142304907" watchObservedRunningTime="2026-07-28 12:07:59.820422539 +0000 UTC m=+114.145087817" Jul 28 12:08:04.576608 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:04.576572 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:08:12.785708 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:12.785648 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-54f6966796-mjljd" podUID="0e679f3e-80c4-41f3-9f67-8d037dacddc3" containerName="console" containerID="cri-o://fd0d603ae4f2efb5a4eb52668296a4039d63df7f9d978d747fd760429a65d628" gracePeriod=15 Jul 28 12:08:12.843814 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:12.843788 2575 generic.go:356] "Generic (PLEG): container finished" podID="0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7" containerID="0f0c24f1665adb0bc4dea636a0a495a6c8cab63ad0a8fec2c62bb75f4ffebf62" exitCode=0 Jul 28 12:08:12.843908 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:12.843845 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h" event={"ID":"0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7","Type":"ContainerDied","Data":"0f0c24f1665adb0bc4dea636a0a495a6c8cab63ad0a8fec2c62bb75f4ffebf62"} Jul 28 12:08:12.844109 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:12.844097 2575 scope.go:117] "RemoveContainer" containerID="0f0c24f1665adb0bc4dea636a0a495a6c8cab63ad0a8fec2c62bb75f4ffebf62" Jul 28 12:08:13.081648 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.081626 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:08:13.235478 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.235454 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-oauth-serving-cert\") pod \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " Jul 28 12:08:13.235592 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.235502 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-serving-cert\") pod \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " Jul 28 12:08:13.235592 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.235523 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-oauth-config\") pod \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " Jul 28 12:08:13.235592 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.235571 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-service-ca\") pod \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " Jul 28 12:08:13.235592 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.235588 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-config\") pod \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " Jul 28 12:08:13.235766 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.235638 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4j9g\" (UniqueName: \"kubernetes.io/projected/0e679f3e-80c4-41f3-9f67-8d037dacddc3-kube-api-access-f4j9g\") pod \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\" (UID: \"0e679f3e-80c4-41f3-9f67-8d037dacddc3\") " Jul 28 12:08:13.235948 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.235915 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-service-ca" (OuterVolumeSpecName: "service-ca") pod "0e679f3e-80c4-41f3-9f67-8d037dacddc3" (UID: "0e679f3e-80c4-41f3-9f67-8d037dacddc3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:08:13.235948 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.235857 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0e679f3e-80c4-41f3-9f67-8d037dacddc3" (UID: "0e679f3e-80c4-41f3-9f67-8d037dacddc3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:08:13.236067 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.236002 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-config" (OuterVolumeSpecName: "console-config") pod "0e679f3e-80c4-41f3-9f67-8d037dacddc3" (UID: "0e679f3e-80c4-41f3-9f67-8d037dacddc3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:08:13.237692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.237665 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0e679f3e-80c4-41f3-9f67-8d037dacddc3" (UID: "0e679f3e-80c4-41f3-9f67-8d037dacddc3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:08:13.237821 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.237802 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0e679f3e-80c4-41f3-9f67-8d037dacddc3" (UID: "0e679f3e-80c4-41f3-9f67-8d037dacddc3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:08:13.237872 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.237813 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e679f3e-80c4-41f3-9f67-8d037dacddc3-kube-api-access-f4j9g" (OuterVolumeSpecName: "kube-api-access-f4j9g") pod "0e679f3e-80c4-41f3-9f67-8d037dacddc3" (UID: "0e679f3e-80c4-41f3-9f67-8d037dacddc3"). InnerVolumeSpecName "kube-api-access-f4j9g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 12:08:13.336340 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.336271 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-serving-cert\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:08:13.336340 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.336295 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-oauth-config\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:08:13.336340 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.336321 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-service-ca\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:08:13.336340 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.336335 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-console-config\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:08:13.336509 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.336347 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4j9g\" (UniqueName: \"kubernetes.io/projected/0e679f3e-80c4-41f3-9f67-8d037dacddc3-kube-api-access-f4j9g\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:08:13.336509 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.336360 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e679f3e-80c4-41f3-9f67-8d037dacddc3-oauth-serving-cert\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:08:13.850348 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.850290 2575 generic.go:356] "Generic (PLEG): container finished" podID="0e679f3e-80c4-41f3-9f67-8d037dacddc3" containerID="fd0d603ae4f2efb5a4eb52668296a4039d63df7f9d978d747fd760429a65d628" exitCode=0 Jul 28 12:08:13.850804 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.850355 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f6966796-mjljd" Jul 28 12:08:13.850804 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.850385 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f6966796-mjljd" event={"ID":"0e679f3e-80c4-41f3-9f67-8d037dacddc3","Type":"ContainerDied","Data":"fd0d603ae4f2efb5a4eb52668296a4039d63df7f9d978d747fd760429a65d628"} Jul 28 12:08:13.850804 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.850444 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f6966796-mjljd" event={"ID":"0e679f3e-80c4-41f3-9f67-8d037dacddc3","Type":"ContainerDied","Data":"e7c7ca650b93e7ba5043922ab11bc4b16a36e5b46ae81ee3dd4654bb2b6cb9ef"} Jul 28 12:08:13.850804 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.850474 2575 scope.go:117] "RemoveContainer" containerID="fd0d603ae4f2efb5a4eb52668296a4039d63df7f9d978d747fd760429a65d628" Jul 28 12:08:13.852369 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.852345 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-xp72h" event={"ID":"0b4caacf-830c-46b5-8b7a-5d0b06a9f7c7","Type":"ContainerStarted","Data":"5d48841ee57f1344c035c0be3f3c35d4921f410f8171f49f2a3181d24c26f3c9"} Jul 28 12:08:13.859266 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.859251 2575 scope.go:117] "RemoveContainer" containerID="fd0d603ae4f2efb5a4eb52668296a4039d63df7f9d978d747fd760429a65d628" Jul 28 12:08:13.859539 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:08:13.859514 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd0d603ae4f2efb5a4eb52668296a4039d63df7f9d978d747fd760429a65d628\": container with ID starting with fd0d603ae4f2efb5a4eb52668296a4039d63df7f9d978d747fd760429a65d628 not found: ID does not exist" containerID="fd0d603ae4f2efb5a4eb52668296a4039d63df7f9d978d747fd760429a65d628" Jul 28 12:08:13.859598 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.859549 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd0d603ae4f2efb5a4eb52668296a4039d63df7f9d978d747fd760429a65d628"} err="failed to get container status \"fd0d603ae4f2efb5a4eb52668296a4039d63df7f9d978d747fd760429a65d628\": rpc error: code = NotFound desc = could not find container \"fd0d603ae4f2efb5a4eb52668296a4039d63df7f9d978d747fd760429a65d628\": container with ID starting with fd0d603ae4f2efb5a4eb52668296a4039d63df7f9d978d747fd760429a65d628 not found: ID does not exist" Jul 28 12:08:13.891462 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.891438 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54f6966796-mjljd"] Jul 28 12:08:13.897211 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:13.897181 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54f6966796-mjljd"] Jul 28 12:08:14.209256 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:14.209229 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e679f3e-80c4-41f3-9f67-8d037dacddc3" path="/var/lib/kubelet/pods/0e679f3e-80c4-41f3-9f67-8d037dacddc3/volumes" Jul 28 12:08:19.957583 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:19.957536 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-97cf9747f-99xkg" podUID="c78be9ee-ce1e-49e3-b532-470765378076" containerName="console" containerID="cri-o://38f2a40f4164b090a8c1e8dd09a97bdb375046ebf964df17f18d671867e76169" gracePeriod=15 Jul 28 12:08:20.232399 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.232337 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:08:20.392952 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.392922 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-oauth-serving-cert\") pod \"c78be9ee-ce1e-49e3-b532-470765378076\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " Jul 28 12:08:20.393131 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.392988 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-trusted-ca-bundle\") pod \"c78be9ee-ce1e-49e3-b532-470765378076\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " Jul 28 12:08:20.393131 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.393099 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-service-ca\") pod \"c78be9ee-ce1e-49e3-b532-470765378076\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " Jul 28 12:08:20.393251 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.393193 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c78be9ee-ce1e-49e3-b532-470765378076-console-oauth-config\") pod \"c78be9ee-ce1e-49e3-b532-470765378076\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " Jul 28 12:08:20.393251 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.393231 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-console-config\") pod \"c78be9ee-ce1e-49e3-b532-470765378076\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " Jul 28 12:08:20.393393 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.393259 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c78be9ee-ce1e-49e3-b532-470765378076-console-serving-cert\") pod \"c78be9ee-ce1e-49e3-b532-470765378076\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " Jul 28 12:08:20.393393 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.393285 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sd59h\" (UniqueName: \"kubernetes.io/projected/c78be9ee-ce1e-49e3-b532-470765378076-kube-api-access-sd59h\") pod \"c78be9ee-ce1e-49e3-b532-470765378076\" (UID: \"c78be9ee-ce1e-49e3-b532-470765378076\") " Jul 28 12:08:20.393393 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.393366 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c78be9ee-ce1e-49e3-b532-470765378076" (UID: "c78be9ee-ce1e-49e3-b532-470765378076"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:08:20.393393 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.393379 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c78be9ee-ce1e-49e3-b532-470765378076" (UID: "c78be9ee-ce1e-49e3-b532-470765378076"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:08:20.393617 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.393417 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-service-ca" (OuterVolumeSpecName: "service-ca") pod "c78be9ee-ce1e-49e3-b532-470765378076" (UID: "c78be9ee-ce1e-49e3-b532-470765378076"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:08:20.393668 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.393623 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-oauth-serving-cert\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:08:20.393668 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.393641 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-trusted-ca-bundle\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:08:20.393668 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.393654 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-service-ca\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:08:20.393817 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.393793 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-console-config" (OuterVolumeSpecName: "console-config") pod "c78be9ee-ce1e-49e3-b532-470765378076" (UID: "c78be9ee-ce1e-49e3-b532-470765378076"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:08:20.395682 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.395642 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c78be9ee-ce1e-49e3-b532-470765378076-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c78be9ee-ce1e-49e3-b532-470765378076" (UID: "c78be9ee-ce1e-49e3-b532-470765378076"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:08:20.395947 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.395928 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c78be9ee-ce1e-49e3-b532-470765378076-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c78be9ee-ce1e-49e3-b532-470765378076" (UID: "c78be9ee-ce1e-49e3-b532-470765378076"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:08:20.396040 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.395945 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c78be9ee-ce1e-49e3-b532-470765378076-kube-api-access-sd59h" (OuterVolumeSpecName: "kube-api-access-sd59h") pod "c78be9ee-ce1e-49e3-b532-470765378076" (UID: "c78be9ee-ce1e-49e3-b532-470765378076"). InnerVolumeSpecName "kube-api-access-sd59h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 12:08:20.494093 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.494044 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c78be9ee-ce1e-49e3-b532-470765378076-console-oauth-config\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:08:20.494093 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.494065 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c78be9ee-ce1e-49e3-b532-470765378076-console-config\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:08:20.494093 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.494075 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c78be9ee-ce1e-49e3-b532-470765378076-console-serving-cert\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:08:20.494093 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.494084 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sd59h\" (UniqueName: \"kubernetes.io/projected/c78be9ee-ce1e-49e3-b532-470765378076-kube-api-access-sd59h\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:08:20.887621 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.887533 2575 generic.go:356] "Generic (PLEG): container finished" podID="c78be9ee-ce1e-49e3-b532-470765378076" containerID="38f2a40f4164b090a8c1e8dd09a97bdb375046ebf964df17f18d671867e76169" exitCode=0 Jul 28 12:08:20.887621 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.887569 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-97cf9747f-99xkg" Jul 28 12:08:20.887621 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.887576 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-97cf9747f-99xkg" event={"ID":"c78be9ee-ce1e-49e3-b532-470765378076","Type":"ContainerDied","Data":"38f2a40f4164b090a8c1e8dd09a97bdb375046ebf964df17f18d671867e76169"} Jul 28 12:08:20.887884 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.887624 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-97cf9747f-99xkg" event={"ID":"c78be9ee-ce1e-49e3-b532-470765378076","Type":"ContainerDied","Data":"b945d75d8904cdf633265a4cb72ade24148aff6b77abfe4568c5442a9852b780"} Jul 28 12:08:20.887884 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.887651 2575 scope.go:117] "RemoveContainer" containerID="38f2a40f4164b090a8c1e8dd09a97bdb375046ebf964df17f18d671867e76169" Jul 28 12:08:20.896179 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.896158 2575 scope.go:117] "RemoveContainer" containerID="38f2a40f4164b090a8c1e8dd09a97bdb375046ebf964df17f18d671867e76169" Jul 28 12:08:20.896448 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:08:20.896426 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38f2a40f4164b090a8c1e8dd09a97bdb375046ebf964df17f18d671867e76169\": container with ID starting with 38f2a40f4164b090a8c1e8dd09a97bdb375046ebf964df17f18d671867e76169 not found: ID does not exist" containerID="38f2a40f4164b090a8c1e8dd09a97bdb375046ebf964df17f18d671867e76169" Jul 28 12:08:20.896526 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.896461 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38f2a40f4164b090a8c1e8dd09a97bdb375046ebf964df17f18d671867e76169"} err="failed to get container status \"38f2a40f4164b090a8c1e8dd09a97bdb375046ebf964df17f18d671867e76169\": rpc error: code = NotFound desc = could not find container \"38f2a40f4164b090a8c1e8dd09a97bdb375046ebf964df17f18d671867e76169\": container with ID starting with 38f2a40f4164b090a8c1e8dd09a97bdb375046ebf964df17f18d671867e76169 not found: ID does not exist" Jul 28 12:08:20.910600 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.910575 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-97cf9747f-99xkg"] Jul 28 12:08:20.914012 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:20.913991 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-97cf9747f-99xkg"] Jul 28 12:08:22.208819 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:22.208785 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c78be9ee-ce1e-49e3-b532-470765378076" path="/var/lib/kubelet/pods/c78be9ee-ce1e-49e3-b532-470765378076/volumes" Jul 28 12:08:49.576460 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:49.576420 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:08:49.592109 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:49.592073 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:08:50.036958 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:08:50.036930 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:09:07.987126 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:07.987091 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 12:09:07.987572 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:07.987398 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="prometheus" containerID="cri-o://74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce" gracePeriod=600 Jul 28 12:09:07.987572 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:07.987418 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276" gracePeriod=600 Jul 28 12:09:07.987572 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:07.987440 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="kube-rbac-proxy-web" containerID="cri-o://454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc" gracePeriod=600 Jul 28 12:09:07.987572 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:07.987460 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="thanos-sidecar" containerID="cri-o://98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d" gracePeriod=600 Jul 28 12:09:07.987572 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:07.987489 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="kube-rbac-proxy" containerID="cri-o://0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1" gracePeriod=600 Jul 28 12:09:07.987572 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:07.987512 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="config-reloader" containerID="cri-o://be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657" gracePeriod=600 Jul 28 12:09:08.238108 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.238044 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:09:08.366145 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366101 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-kube-rbac-proxy\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366145 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366148 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-tls\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366413 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366166 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-2\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366413 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366188 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-trusted-ca-bundle\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366413 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366231 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-thanos-prometheus-http-client-file\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366413 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366258 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-kubelet-serving-ca-bundle\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366413 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366291 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lc8gv\" (UniqueName: \"kubernetes.io/projected/28bd0d01-e07e-44c0-be06-886a156d2a6d-kube-api-access-lc8gv\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366413 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366345 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-metrics-client-certs\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366413 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366385 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-metrics-client-ca\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366413 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366410 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366806 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366435 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-serving-certs-ca-bundle\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366806 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366470 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28bd0d01-e07e-44c0-be06-886a156d2a6d-config-out\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366806 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366536 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-1\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366806 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366543 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:09:08.366806 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366562 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366806 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366600 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-web-config\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366806 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366624 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-grpc-tls\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366806 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366661 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28bd0d01-e07e-44c0-be06-886a156d2a6d-tls-assets\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366806 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366691 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:09:08.366806 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366717 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-config\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366806 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366696 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:09:08.366806 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366745 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-db\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.366806 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.366777 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-0\") pod \"28bd0d01-e07e-44c0-be06-886a156d2a6d\" (UID: \"28bd0d01-e07e-44c0-be06-886a156d2a6d\") " Jul 28 12:09:08.367482 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.367052 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.367482 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.367070 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.367482 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.367085 2575 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.367641 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.367527 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:09:08.368605 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.368000 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:09:08.368605 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.368356 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:09:08.368605 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.368582 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:09:08.368830 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.368805 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 12:09:08.369215 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.369160 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:09:08.370010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.369979 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:09:08.370473 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.370445 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-config" (OuterVolumeSpecName: "config") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:09:08.370599 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.370493 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:09:08.370670 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.370611 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:09:08.370751 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.370729 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:09:08.370923 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.370896 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28bd0d01-e07e-44c0-be06-886a156d2a6d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 12:09:08.371029 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.370995 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:09:08.371517 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.371497 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:09:08.371857 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.371841 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28bd0d01-e07e-44c0-be06-886a156d2a6d-config-out" (OuterVolumeSpecName: "config-out") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 12:09:08.372256 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.372239 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28bd0d01-e07e-44c0-be06-886a156d2a6d-kube-api-access-lc8gv" (OuterVolumeSpecName: "kube-api-access-lc8gv") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "kube-api-access-lc8gv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 12:09:08.381597 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.381577 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-web-config" (OuterVolumeSpecName: "web-config") pod "28bd0d01-e07e-44c0-be06-886a156d2a6d" (UID: "28bd0d01-e07e-44c0-be06-886a156d2a6d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:09:08.467867 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467830 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.467867 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467860 2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.467867 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467871 2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-web-config\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.468089 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467880 2575 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-grpc-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.468089 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467889 2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28bd0d01-e07e-44c0-be06-886a156d2a6d-tls-assets\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.468089 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467896 2575 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-config\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.468089 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467904 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-db\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.468089 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467912 2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.468089 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467920 2575 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-kube-rbac-proxy\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.468089 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467930 2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.468089 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467938 2575 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.468089 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467947 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lc8gv\" (UniqueName: \"kubernetes.io/projected/28bd0d01-e07e-44c0-be06-886a156d2a6d-kube-api-access-lc8gv\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.468089 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467956 2575 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-metrics-client-certs\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.468089 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467965 2575 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-metrics-client-ca\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.468089 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467974 2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/28bd0d01-e07e-44c0-be06-886a156d2a6d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.468089 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467984 2575 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bd0d01-e07e-44c0-be06-886a156d2a6d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:08.468089 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:08.467992 2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28bd0d01-e07e-44c0-be06-886a156d2a6d-config-out\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:09:09.100156 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.100121 2575 generic.go:356] "Generic (PLEG): container finished" podID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerID="c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276" exitCode=0 Jul 28 12:09:09.100156 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.100150 2575 generic.go:356] "Generic (PLEG): container finished" podID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerID="0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1" exitCode=0 Jul 28 12:09:09.100156 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.100160 2575 generic.go:356] "Generic (PLEG): container finished" podID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerID="454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc" exitCode=0 Jul 28 12:09:09.100737 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.100170 2575 generic.go:356] "Generic (PLEG): container finished" podID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerID="98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d" exitCode=0 Jul 28 12:09:09.100737 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.100178 2575 generic.go:356] "Generic (PLEG): container finished" podID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerID="be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657" exitCode=0 Jul 28 12:09:09.100737 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.100186 2575 generic.go:356] "Generic (PLEG): container finished" podID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerID="74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce" exitCode=0 Jul 28 12:09:09.100737 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.100210 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 12:09:09.100737 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.100218 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bd0d01-e07e-44c0-be06-886a156d2a6d","Type":"ContainerDied","Data":"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276"} Jul 28 12:09:09.100737 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.100301 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bd0d01-e07e-44c0-be06-886a156d2a6d","Type":"ContainerDied","Data":"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1"} Jul 28 12:09:09.100737 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.100341 2575 scope.go:117] "RemoveContainer" containerID="c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276" Jul 28 12:09:09.100737 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.100364 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bd0d01-e07e-44c0-be06-886a156d2a6d","Type":"ContainerDied","Data":"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc"} Jul 28 12:09:09.100737 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.100390 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bd0d01-e07e-44c0-be06-886a156d2a6d","Type":"ContainerDied","Data":"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d"} Jul 28 12:09:09.100737 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.100418 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bd0d01-e07e-44c0-be06-886a156d2a6d","Type":"ContainerDied","Data":"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657"} Jul 28 12:09:09.100737 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.100442 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bd0d01-e07e-44c0-be06-886a156d2a6d","Type":"ContainerDied","Data":"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce"} Jul 28 12:09:09.100737 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.100469 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bd0d01-e07e-44c0-be06-886a156d2a6d","Type":"ContainerDied","Data":"f2df582d80c1d3d0e119e1412d1578f07a068a444cf0c9c62fea27bb18f30c3e"} Jul 28 12:09:09.110569 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.110541 2575 scope.go:117] "RemoveContainer" containerID="0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1" Jul 28 12:09:09.122574 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.122548 2575 scope.go:117] "RemoveContainer" containerID="454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc" Jul 28 12:09:09.129739 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.129722 2575 scope.go:117] "RemoveContainer" containerID="98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d" Jul 28 12:09:09.136573 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.136555 2575 scope.go:117] "RemoveContainer" containerID="be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657" Jul 28 12:09:09.140459 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.140437 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 12:09:09.144545 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.144519 2575 scope.go:117] "RemoveContainer" containerID="74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce" Jul 28 12:09:09.146534 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.146514 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 12:09:09.152413 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.152393 2575 scope.go:117] "RemoveContainer" containerID="9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f" Jul 28 12:09:09.159007 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.158988 2575 scope.go:117] "RemoveContainer" containerID="c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276" Jul 28 12:09:09.159246 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:09:09.159226 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276\": container with ID starting with c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276 not found: ID does not exist" containerID="c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276" Jul 28 12:09:09.159329 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.159254 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276"} err="failed to get container status \"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276\": rpc error: code = NotFound desc = could not find container \"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276\": container with ID starting with c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276 not found: ID does not exist" Jul 28 12:09:09.159329 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.159273 2575 scope.go:117] "RemoveContainer" containerID="0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1" Jul 28 12:09:09.159525 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:09:09.159508 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1\": container with ID starting with 0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1 not found: ID does not exist" containerID="0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1" Jul 28 12:09:09.159560 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.159531 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1"} err="failed to get container status \"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1\": rpc error: code = NotFound desc = could not find container \"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1\": container with ID starting with 0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1 not found: ID does not exist" Jul 28 12:09:09.159560 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.159546 2575 scope.go:117] "RemoveContainer" containerID="454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc" Jul 28 12:09:09.159767 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:09:09.159751 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc\": container with ID starting with 454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc not found: ID does not exist" containerID="454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc" Jul 28 12:09:09.159804 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.159771 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc"} err="failed to get container status \"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc\": rpc error: code = NotFound desc = could not find container \"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc\": container with ID starting with 454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc not found: ID does not exist" Jul 28 12:09:09.159804 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.159785 2575 scope.go:117] "RemoveContainer" containerID="98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d" Jul 28 12:09:09.159971 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:09:09.159957 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d\": container with ID starting with 98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d not found: ID does not exist" containerID="98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d" Jul 28 12:09:09.160014 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.159974 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d"} err="failed to get container status \"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d\": rpc error: code = NotFound desc = could not find container \"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d\": container with ID starting with 98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d not found: ID does not exist" Jul 28 12:09:09.160014 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.159987 2575 scope.go:117] "RemoveContainer" containerID="be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657" Jul 28 12:09:09.160178 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:09:09.160166 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657\": container with ID starting with be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657 not found: ID does not exist" containerID="be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657" Jul 28 12:09:09.160225 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.160180 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657"} err="failed to get container status \"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657\": rpc error: code = NotFound desc = could not find container \"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657\": container with ID starting with be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657 not found: ID does not exist" Jul 28 12:09:09.160225 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.160192 2575 scope.go:117] "RemoveContainer" containerID="74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce" Jul 28 12:09:09.160390 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:09:09.160374 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce\": container with ID starting with 74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce not found: ID does not exist" containerID="74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce" Jul 28 12:09:09.160435 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.160393 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce"} err="failed to get container status \"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce\": rpc error: code = NotFound desc = could not find container \"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce\": container with ID starting with 74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce not found: ID does not exist" Jul 28 12:09:09.160435 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.160406 2575 scope.go:117] "RemoveContainer" containerID="9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f" Jul 28 12:09:09.160646 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:09:09.160631 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f\": container with ID starting with 9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f not found: ID does not exist" containerID="9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f" Jul 28 12:09:09.160681 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.160648 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f"} err="failed to get container status \"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f\": rpc error: code = NotFound desc = could not find container \"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f\": container with ID starting with 9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f not found: ID does not exist" Jul 28 12:09:09.160681 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.160659 2575 scope.go:117] "RemoveContainer" containerID="c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276" Jul 28 12:09:09.160886 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.160870 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276"} err="failed to get container status \"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276\": rpc error: code = NotFound desc = could not find container \"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276\": container with ID starting with c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276 not found: ID does not exist" Jul 28 12:09:09.160930 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.160887 2575 scope.go:117] "RemoveContainer" containerID="0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1" Jul 28 12:09:09.161064 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.161047 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1"} err="failed to get container status \"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1\": rpc error: code = NotFound desc = could not find container \"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1\": container with ID starting with 0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1 not found: ID does not exist" Jul 28 12:09:09.161104 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.161065 2575 scope.go:117] "RemoveContainer" containerID="454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc" Jul 28 12:09:09.161212 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.161198 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc"} err="failed to get container status \"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc\": rpc error: code = NotFound desc = could not find container \"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc\": container with ID starting with 454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc not found: ID does not exist" Jul 28 12:09:09.161249 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.161212 2575 scope.go:117] "RemoveContainer" containerID="98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d" Jul 28 12:09:09.161473 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.161453 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d"} err="failed to get container status \"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d\": rpc error: code = NotFound desc = could not find container \"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d\": container with ID starting with 98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d not found: ID does not exist" Jul 28 12:09:09.161523 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.161473 2575 scope.go:117] "RemoveContainer" containerID="be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657" Jul 28 12:09:09.161668 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.161652 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657"} err="failed to get container status \"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657\": rpc error: code = NotFound desc = could not find container \"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657\": container with ID starting with be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657 not found: ID does not exist" Jul 28 12:09:09.161716 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.161669 2575 scope.go:117] "RemoveContainer" containerID="74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce" Jul 28 12:09:09.161988 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.161954 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce"} err="failed to get container status \"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce\": rpc error: code = NotFound desc = could not find container \"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce\": container with ID starting with 74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce not found: ID does not exist" Jul 28 12:09:09.161988 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.161978 2575 scope.go:117] "RemoveContainer" containerID="9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f" Jul 28 12:09:09.162236 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.162213 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f"} err="failed to get container status \"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f\": rpc error: code = NotFound desc = could not find container \"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f\": container with ID starting with 9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f not found: ID does not exist" Jul 28 12:09:09.162236 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.162236 2575 scope.go:117] "RemoveContainer" containerID="c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276" Jul 28 12:09:09.162496 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.162476 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276"} err="failed to get container status \"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276\": rpc error: code = NotFound desc = could not find container \"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276\": container with ID starting with c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276 not found: ID does not exist" Jul 28 12:09:09.162557 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.162498 2575 scope.go:117] "RemoveContainer" containerID="0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1" Jul 28 12:09:09.162744 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.162716 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1"} err="failed to get container status \"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1\": rpc error: code = NotFound desc = could not find container \"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1\": container with ID starting with 0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1 not found: ID does not exist" Jul 28 12:09:09.162744 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.162744 2575 scope.go:117] "RemoveContainer" containerID="454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc" Jul 28 12:09:09.163031 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.162999 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc"} err="failed to get container status \"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc\": rpc error: code = NotFound desc = could not find container \"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc\": container with ID starting with 454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc not found: ID does not exist" Jul 28 12:09:09.163031 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.163024 2575 scope.go:117] "RemoveContainer" containerID="98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d" Jul 28 12:09:09.163372 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.163294 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d"} err="failed to get container status \"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d\": rpc error: code = NotFound desc = could not find container \"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d\": container with ID starting with 98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d not found: ID does not exist" Jul 28 12:09:09.163372 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.163334 2575 scope.go:117] "RemoveContainer" containerID="be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657" Jul 28 12:09:09.163751 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.163723 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657"} err="failed to get container status \"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657\": rpc error: code = NotFound desc = could not find container \"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657\": container with ID starting with be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657 not found: ID does not exist" Jul 28 12:09:09.163751 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.163750 2575 scope.go:117] "RemoveContainer" containerID="74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce" Jul 28 12:09:09.165264 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.165241 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce"} err="failed to get container status \"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce\": rpc error: code = NotFound desc = could not find container \"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce\": container with ID starting with 74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce not found: ID does not exist" Jul 28 12:09:09.165264 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.165263 2575 scope.go:117] "RemoveContainer" containerID="9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f" Jul 28 12:09:09.165571 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.165539 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f"} err="failed to get container status \"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f\": rpc error: code = NotFound desc = could not find container \"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f\": container with ID starting with 9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f not found: ID does not exist" Jul 28 12:09:09.165618 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.165573 2575 scope.go:117] "RemoveContainer" containerID="c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276" Jul 28 12:09:09.165861 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.165823 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276"} err="failed to get container status \"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276\": rpc error: code = NotFound desc = could not find container \"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276\": container with ID starting with c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276 not found: ID does not exist" Jul 28 12:09:09.165861 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.165855 2575 scope.go:117] "RemoveContainer" containerID="0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1" Jul 28 12:09:09.166124 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.166103 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1"} err="failed to get container status \"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1\": rpc error: code = NotFound desc = could not find container \"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1\": container with ID starting with 0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1 not found: ID does not exist" Jul 28 12:09:09.166183 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.166125 2575 scope.go:117] "RemoveContainer" containerID="454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc" Jul 28 12:09:09.166382 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.166364 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc"} err="failed to get container status \"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc\": rpc error: code = NotFound desc = could not find container \"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc\": container with ID starting with 454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc not found: ID does not exist" Jul 28 12:09:09.166444 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.166392 2575 scope.go:117] "RemoveContainer" containerID="98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d" Jul 28 12:09:09.166616 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.166600 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d"} err="failed to get container status \"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d\": rpc error: code = NotFound desc = could not find container \"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d\": container with ID starting with 98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d not found: ID does not exist" Jul 28 12:09:09.166657 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.166617 2575 scope.go:117] "RemoveContainer" containerID="be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657" Jul 28 12:09:09.166810 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.166791 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657"} err="failed to get container status \"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657\": rpc error: code = NotFound desc = could not find container \"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657\": container with ID starting with be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657 not found: ID does not exist" Jul 28 12:09:09.166864 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.166811 2575 scope.go:117] "RemoveContainer" containerID="74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce" Jul 28 12:09:09.167015 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.166999 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce"} err="failed to get container status \"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce\": rpc error: code = NotFound desc = could not find container \"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce\": container with ID starting with 74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce not found: ID does not exist" Jul 28 12:09:09.167084 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.167016 2575 scope.go:117] "RemoveContainer" containerID="9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f" Jul 28 12:09:09.167374 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.167296 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f"} err="failed to get container status \"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f\": rpc error: code = NotFound desc = could not find container \"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f\": container with ID starting with 9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f not found: ID does not exist" Jul 28 12:09:09.167374 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.167339 2575 scope.go:117] "RemoveContainer" containerID="c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276" Jul 28 12:09:09.167600 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.167580 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276"} err="failed to get container status \"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276\": rpc error: code = NotFound desc = could not find container \"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276\": container with ID starting with c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276 not found: ID does not exist" Jul 28 12:09:09.167663 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.167602 2575 scope.go:117] "RemoveContainer" containerID="0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1" Jul 28 12:09:09.167852 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.167834 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1"} err="failed to get container status \"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1\": rpc error: code = NotFound desc = could not find container \"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1\": container with ID starting with 0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1 not found: ID does not exist" Jul 28 12:09:09.167929 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.167852 2575 scope.go:117] "RemoveContainer" containerID="454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc" Jul 28 12:09:09.168071 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.168055 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc"} err="failed to get container status \"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc\": rpc error: code = NotFound desc = could not find container \"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc\": container with ID starting with 454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc not found: ID does not exist" Jul 28 12:09:09.168135 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.168083 2575 scope.go:117] "RemoveContainer" containerID="98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d" Jul 28 12:09:09.168337 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.168317 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d"} err="failed to get container status \"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d\": rpc error: code = NotFound desc = could not find container \"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d\": container with ID starting with 98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d not found: ID does not exist" Jul 28 12:09:09.168379 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.168338 2575 scope.go:117] "RemoveContainer" containerID="be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657" Jul 28 12:09:09.168547 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.168530 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657"} err="failed to get container status \"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657\": rpc error: code = NotFound desc = could not find container \"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657\": container with ID starting with be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657 not found: ID does not exist" Jul 28 12:09:09.168612 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.168549 2575 scope.go:117] "RemoveContainer" containerID="74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce" Jul 28 12:09:09.168761 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.168743 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce"} err="failed to get container status \"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce\": rpc error: code = NotFound desc = could not find container \"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce\": container with ID starting with 74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce not found: ID does not exist" Jul 28 12:09:09.168805 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.168762 2575 scope.go:117] "RemoveContainer" containerID="9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f" Jul 28 12:09:09.168938 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.168922 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f"} err="failed to get container status \"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f\": rpc error: code = NotFound desc = could not find container \"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f\": container with ID starting with 9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f not found: ID does not exist" Jul 28 12:09:09.169003 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.168939 2575 scope.go:117] "RemoveContainer" containerID="c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276" Jul 28 12:09:09.169179 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.169161 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276"} err="failed to get container status \"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276\": rpc error: code = NotFound desc = could not find container \"c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276\": container with ID starting with c67f58070298a7ca02503a559f215be049ff420a27d16076f7929019fe6f9276 not found: ID does not exist" Jul 28 12:09:09.169257 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.169179 2575 scope.go:117] "RemoveContainer" containerID="0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1" Jul 28 12:09:09.169401 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.169384 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1"} err="failed to get container status \"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1\": rpc error: code = NotFound desc = could not find container \"0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1\": container with ID starting with 0f77a033b4e1c4e560d00833b04478900b289f17439c61ec2f15b5526108b4a1 not found: ID does not exist" Jul 28 12:09:09.169451 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.169403 2575 scope.go:117] "RemoveContainer" containerID="454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc" Jul 28 12:09:09.169626 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.169608 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc"} err="failed to get container status \"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc\": rpc error: code = NotFound desc = could not find container \"454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc\": container with ID starting with 454ef4027f7526f2873af6db2d474a2ffd08fccb44f9554019f69db875b351bc not found: ID does not exist" Jul 28 12:09:09.169626 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.169626 2575 scope.go:117] "RemoveContainer" containerID="98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d" Jul 28 12:09:09.169845 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.169830 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d"} err="failed to get container status \"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d\": rpc error: code = NotFound desc = could not find container \"98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d\": container with ID starting with 98e1a8d8a784357162e639c0345fad47552fe9517367d156b54fe3560a6d247d not found: ID does not exist" Jul 28 12:09:09.169889 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.169846 2575 scope.go:117] "RemoveContainer" containerID="be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657" Jul 28 12:09:09.170053 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.170035 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657"} err="failed to get container status \"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657\": rpc error: code = NotFound desc = could not find container \"be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657\": container with ID starting with be0aab8c00b84da7bcd148b8c932251915a4e39864eacdd3e95a10fbc8289657 not found: ID does not exist" Jul 28 12:09:09.170120 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.170054 2575 scope.go:117] "RemoveContainer" containerID="74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce" Jul 28 12:09:09.170332 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.170293 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce"} err="failed to get container status \"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce\": rpc error: code = NotFound desc = could not find container \"74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce\": container with ID starting with 74b1bf88180168dbea2960456437654a596a55e6d7881fa5803a9f19e900f8ce not found: ID does not exist" Jul 28 12:09:09.170398 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.170333 2575 scope.go:117] "RemoveContainer" containerID="9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f" Jul 28 12:09:09.170544 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:09.170525 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f"} err="failed to get container status \"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f\": rpc error: code = NotFound desc = could not find container \"9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f\": container with ID starting with 9555b701daa9d7c69e2a9d41962edcef61ad86e7b5fcebb6f6bad382fca0064f not found: ID does not exist" Jul 28 12:09:10.209125 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:09:10.209090 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" path="/var/lib/kubelet/pods/28bd0d01-e07e-44c0-be06-886a156d2a6d/volumes" Jul 28 12:11:06.066564 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:06.066533 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/1.log" Jul 28 12:11:06.073331 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:06.073292 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/1.log" Jul 28 12:11:06.076718 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:06.076694 2575 kubelet.go:1648] "Image garbage collection succeeded" Jul 28 12:11:47.861214 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861179 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7"] Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861478 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="prometheus" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861489 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="prometheus" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861501 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e679f3e-80c4-41f3-9f67-8d037dacddc3" containerName="console" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861506 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e679f3e-80c4-41f3-9f67-8d037dacddc3" containerName="console" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861516 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="config-reloader" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861521 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="config-reloader" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861529 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="init-config-reloader" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861534 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="init-config-reloader" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861541 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="kube-rbac-proxy" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861546 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="kube-rbac-proxy" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861558 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c78be9ee-ce1e-49e3-b532-470765378076" containerName="console" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861563 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c78be9ee-ce1e-49e3-b532-470765378076" containerName="console" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861569 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="kube-rbac-proxy-web" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861574 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="kube-rbac-proxy-web" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861579 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="kube-rbac-proxy-thanos" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861584 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="kube-rbac-proxy-thanos" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861593 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="thanos-sidecar" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861598 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="thanos-sidecar" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861644 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="kube-rbac-proxy-web" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861654 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e679f3e-80c4-41f3-9f67-8d037dacddc3" containerName="console" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861661 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="prometheus" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861667 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="thanos-sidecar" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861674 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c78be9ee-ce1e-49e3-b532-470765378076" containerName="console" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861680 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="kube-rbac-proxy-thanos" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861687 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="config-reloader" Jul 28 12:11:47.862692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.861692 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="28bd0d01-e07e-44c0-be06-886a156d2a6d" containerName="kube-rbac-proxy" Jul 28 12:11:47.863564 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.863548 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7" Jul 28 12:11:47.866807 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.866784 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Jul 28 12:11:47.866926 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.866814 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Jul 28 12:11:47.868068 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.868050 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-6tj97\"" Jul 28 12:11:47.923489 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.923466 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7"] Jul 28 12:11:47.992517 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:47.992495 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6p82\" (UniqueName: \"kubernetes.io/projected/6c7a5ac7-2229-4d0c-8934-cd13bceca5cd-kube-api-access-d6p82\") pod \"feast-operator-controller-manager-f68699c4d-qk2h7\" (UID: \"6c7a5ac7-2229-4d0c-8934-cd13bceca5cd\") " pod="feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7" Jul 28 12:11:48.093133 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:48.093109 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6p82\" (UniqueName: \"kubernetes.io/projected/6c7a5ac7-2229-4d0c-8934-cd13bceca5cd-kube-api-access-d6p82\") pod \"feast-operator-controller-manager-f68699c4d-qk2h7\" (UID: \"6c7a5ac7-2229-4d0c-8934-cd13bceca5cd\") " pod="feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7" Jul 28 12:11:48.102482 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:48.102453 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6p82\" (UniqueName: \"kubernetes.io/projected/6c7a5ac7-2229-4d0c-8934-cd13bceca5cd-kube-api-access-d6p82\") pod \"feast-operator-controller-manager-f68699c4d-qk2h7\" (UID: \"6c7a5ac7-2229-4d0c-8934-cd13bceca5cd\") " pod="feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7" Jul 28 12:11:48.174066 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:48.174041 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7" Jul 28 12:11:48.298673 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:48.298648 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7"] Jul 28 12:11:48.300656 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:11:48.300635 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c7a5ac7_2229_4d0c_8934_cd13bceca5cd.slice/crio-fceb5d041f085745699147e11c0cac6ce84f5a7f55fb491bd766a90b3badac14 WatchSource:0}: Error finding container fceb5d041f085745699147e11c0cac6ce84f5a7f55fb491bd766a90b3badac14: Status 404 returned error can't find the container with id fceb5d041f085745699147e11c0cac6ce84f5a7f55fb491bd766a90b3badac14 Jul 28 12:11:48.302981 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:48.302966 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 12:11:48.693028 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:48.692994 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7" event={"ID":"6c7a5ac7-2229-4d0c-8934-cd13bceca5cd","Type":"ContainerStarted","Data":"fceb5d041f085745699147e11c0cac6ce84f5a7f55fb491bd766a90b3badac14"} Jul 28 12:11:50.704396 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:50.704252 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7" event={"ID":"6c7a5ac7-2229-4d0c-8934-cd13bceca5cd","Type":"ContainerStarted","Data":"84c2c103bfc0738e714102ff5ec6a7b27a0cc7a1e94e60ac0a84884d954fad75"} Jul 28 12:11:50.704396 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:50.704381 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7" Jul 28 12:11:50.724004 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:11:50.723953 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7" podStartSLOduration=1.491394232 podStartE2EDuration="3.723938301s" totalImagesPullingTime="2.232544069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:11:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:11:50.723591309 +0000 UTC m=+345.048256587" watchObservedRunningTime="2026-07-28 12:11:50.723938301 +0000 UTC m=+345.048603587" Jul 28 12:12:01.712079 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:01.712041 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7" Jul 28 12:12:28.390428 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.390390 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr"] Jul 28 12:12:28.393080 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.393063 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.396805 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.396779 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-8c77c\"" Jul 28 12:12:28.396946 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.396911 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jul 28 12:12:28.397339 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.397321 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jul 28 12:12:28.397436 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.397346 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Jul 28 12:12:28.397772 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.397758 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jul 28 12:12:28.397865 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.397850 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 28 12:12:28.397903 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.397868 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jul 28 12:12:28.398577 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.398550 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 28 12:12:28.416722 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.416703 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr"] Jul 28 12:12:28.508727 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.508706 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-feast-data\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.508826 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.508737 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-offline-tls\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.508826 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.508755 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v64r\" (UniqueName: \"kubernetes.io/projected/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-kube-api-access-7v64r\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.508902 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.508849 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-online-tls\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.508902 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.508893 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-registry-tls\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.508964 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.508927 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-ui-tls\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.609377 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.609353 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v64r\" (UniqueName: \"kubernetes.io/projected/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-kube-api-access-7v64r\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.609479 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.609397 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-online-tls\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.609479 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.609416 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-registry-tls\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.609479 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.609456 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-ui-tls\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.609607 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.609486 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-feast-data\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.609607 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:12:28.609506 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Jul 28 12:12:28.609607 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.609511 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-offline-tls\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.609607 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:12:28.609575 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-online-tls podName:149d6ee4-fa3d-498a-9ff5-9870f83f12ea nodeName:}" failed. No retries permitted until 2026-07-28 12:12:29.109555259 +0000 UTC m=+383.434220528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-online-tls") pod "feast-simple-feast-setup-66c756fdd-kvbwr" (UID: "149d6ee4-fa3d-498a-9ff5-9870f83f12ea") : secret "feast-simple-feast-setup-online-tls" not found Jul 28 12:12:28.609977 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.609947 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-feast-data\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.611864 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.611840 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-registry-tls\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.612016 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.611997 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-ui-tls\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.612463 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.612443 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-offline-tls\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:28.624732 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:28.624712 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v64r\" (UniqueName: \"kubernetes.io/projected/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-kube-api-access-7v64r\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:29.114627 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:29.114596 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-online-tls\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:29.116893 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:29.116869 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-online-tls\") pod \"feast-simple-feast-setup-66c756fdd-kvbwr\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:29.318900 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:29.318875 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:12:29.470511 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:29.470377 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr"] Jul 28 12:12:29.473008 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:12:29.472975 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod149d6ee4_fa3d_498a_9ff5_9870f83f12ea.slice/crio-1ab0993d44d15d334e99bd5a737533c8c9ad8ed747db8dff7fc328995102e5f8 WatchSource:0}: Error finding container 1ab0993d44d15d334e99bd5a737533c8c9ad8ed747db8dff7fc328995102e5f8: Status 404 returned error can't find the container with id 1ab0993d44d15d334e99bd5a737533c8c9ad8ed747db8dff7fc328995102e5f8 Jul 28 12:12:29.863925 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:29.863895 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" event={"ID":"149d6ee4-fa3d-498a-9ff5-9870f83f12ea","Type":"ContainerStarted","Data":"1ab0993d44d15d334e99bd5a737533c8c9ad8ed747db8dff7fc328995102e5f8"} Jul 28 12:12:58.994476 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:12:58.994434 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" event={"ID":"149d6ee4-fa3d-498a-9ff5-9870f83f12ea","Type":"ContainerStarted","Data":"3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef"} Jul 28 12:13:03.010213 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:03.010136 2575 generic.go:356] "Generic (PLEG): container finished" podID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerID="3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef" exitCode=0 Jul 28 12:13:03.010213 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:03.010195 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" event={"ID":"149d6ee4-fa3d-498a-9ff5-9870f83f12ea","Type":"ContainerDied","Data":"3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef"} Jul 28 12:13:04.016978 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:04.016936 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" event={"ID":"149d6ee4-fa3d-498a-9ff5-9870f83f12ea","Type":"ContainerStarted","Data":"a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3"} Jul 28 12:13:09.040373 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:09.040335 2575 generic.go:356] "Generic (PLEG): container finished" podID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerID="a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3" exitCode=0 Jul 28 12:13:09.040742 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:09.040411 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" event={"ID":"149d6ee4-fa3d-498a-9ff5-9870f83f12ea","Type":"ContainerDied","Data":"a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3"} Jul 28 12:13:09.052999 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:09.052956 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Jul 28 12:13:10.050264 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:10.050191 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" event={"ID":"149d6ee4-fa3d-498a-9ff5-9870f83f12ea","Type":"ContainerStarted","Data":"0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f"} Jul 28 12:13:10.050264 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:10.050254 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" event={"ID":"149d6ee4-fa3d-498a-9ff5-9870f83f12ea","Type":"ContainerStarted","Data":"f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2"} Jul 28 12:13:10.050822 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:10.050275 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" event={"ID":"149d6ee4-fa3d-498a-9ff5-9870f83f12ea","Type":"ContainerStarted","Data":"6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2"} Jul 28 12:13:10.050822 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:10.050294 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" event={"ID":"149d6ee4-fa3d-498a-9ff5-9870f83f12ea","Type":"ContainerStarted","Data":"749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b"} Jul 28 12:13:10.084396 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:10.084332 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podStartSLOduration=13.560521566 podStartE2EDuration="42.084290469s" totalImagesPullingTime="28.523768903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:13:10.081156035 +0000 UTC m=+424.405821313" watchObservedRunningTime="2026-07-28 12:13:10.084290469 +0000 UTC m=+424.408955749" Jul 28 12:13:11.319938 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:11.319905 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:13:11.320517 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:11.320487 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:13:11.320638 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:11.320524 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:13:11.320638 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:11.320540 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:13:11.322237 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:11.322191 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused" Jul 28 12:13:11.322481 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:11.322217 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.27:6571: connect: connection refused" Jul 28 12:13:11.322578 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:11.322214 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.27:8816: connect: connection refused" Jul 28 12:13:11.322578 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:11.322275 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Jul 28 12:13:14.320092 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:14.319942 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.27:8816: connect: connection refused" Jul 28 12:13:14.320092 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:14.319975 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Jul 28 12:13:14.320092 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:14.319942 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.27:6571: connect: connection refused" Jul 28 12:13:14.320717 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:14.320692 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused" Jul 28 12:13:17.320199 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:17.320164 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:13:17.320649 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:17.320508 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:13:17.320960 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:17.320932 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:13:17.321083 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:17.321068 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:13:17.321145 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:17.321099 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:13:17.321200 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:17.321157 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:13:17.321250 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:17.321210 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:13:17.325180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:17.325163 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:13:18.088757 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:18.088728 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:13:18.088944 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:18.088866 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:13:18.089177 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:18.089157 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:13:18.092002 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:13:18.091982 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:14:19.662466 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:19.662428 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr"] Jul 28 12:14:19.662989 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:19.662674 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="registry" containerID="cri-o://749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b" gracePeriod=30 Jul 28 12:14:19.662989 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:19.662732 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="offline" containerID="cri-o://f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2" gracePeriod=30 Jul 28 12:14:19.662989 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:19.662727 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="ui" containerID="cri-o://0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f" gracePeriod=30 Jul 28 12:14:19.662989 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:19.662739 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="online" containerID="cri-o://6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2" gracePeriod=30 Jul 28 12:14:19.944656 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:19.944620 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p"] Jul 28 12:14:19.947687 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:19.947668 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:19.950814 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:19.950796 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-rp94h\"" Jul 28 12:14:19.967421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:19.967391 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p"] Jul 28 12:14:20.038509 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.038471 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-ui-tls\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.038509 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.038511 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnp94\" (UniqueName: \"kubernetes.io/projected/354656be-9109-4915-873f-f945e8959c33-kube-api-access-nnp94\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.038736 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.038543 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-offline-tls\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.038736 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.038596 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-registry-tls\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.038736 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.038637 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-online-tls\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.038736 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.038677 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/354656be-9109-4915-873f-f945e8959c33-feast-data\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.139699 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.139663 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-online-tls\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.139900 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.139716 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/354656be-9109-4915-873f-f945e8959c33-feast-data\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.139900 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.139751 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-ui-tls\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.139900 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.139786 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnp94\" (UniqueName: \"kubernetes.io/projected/354656be-9109-4915-873f-f945e8959c33-kube-api-access-nnp94\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.139900 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.139820 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-offline-tls\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.139900 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.139858 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-registry-tls\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.140154 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:14:20.139903 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Jul 28 12:14:20.140154 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:14:20.139963 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Jul 28 12:14:20.140154 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:14:20.139977 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-ui-tls podName:354656be-9109-4915-873f-f945e8959c33 nodeName:}" failed. No retries permitted until 2026-07-28 12:14:20.639955324 +0000 UTC m=+494.964620582 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-ui-tls") pod "feast-simple-feast-setup-66c756fdd-rbs9p" (UID: "354656be-9109-4915-873f-f945e8959c33") : secret "feast-simple-feast-setup-ui-tls" not found Jul 28 12:14:20.140154 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:14:20.140016 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-registry-tls podName:354656be-9109-4915-873f-f945e8959c33 nodeName:}" failed. No retries permitted until 2026-07-28 12:14:20.639999188 +0000 UTC m=+494.964664451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-registry-tls") pod "feast-simple-feast-setup-66c756fdd-rbs9p" (UID: "354656be-9109-4915-873f-f945e8959c33") : secret "feast-simple-feast-setup-registry-tls" not found Jul 28 12:14:20.140154 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.140144 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/354656be-9109-4915-873f-f945e8959c33-feast-data\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.142353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.142289 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-online-tls\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.142565 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.142540 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-offline-tls\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.150810 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.150782 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnp94\" (UniqueName: \"kubernetes.io/projected/354656be-9109-4915-873f-f945e8959c33-kube-api-access-nnp94\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.342927 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.342901 2575 generic.go:356] "Generic (PLEG): container finished" podID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerID="0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f" exitCode=0 Jul 28 12:14:20.343043 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.342970 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" event={"ID":"149d6ee4-fa3d-498a-9ff5-9870f83f12ea","Type":"ContainerDied","Data":"0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f"} Jul 28 12:14:20.643361 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.643257 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-registry-tls\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.643534 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.643373 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-ui-tls\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.645671 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.645647 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-ui-tls\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.645762 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.645680 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-registry-tls\") pod \"feast-simple-feast-setup-66c756fdd-rbs9p\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.857180 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.857150 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:20.996391 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:20.996366 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p"] Jul 28 12:14:20.997493 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:14:20.997472 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod354656be_9109_4915_873f_f945e8959c33.slice/crio-18ec664fa94a5f0ad6619650b06e1aa23f4cad594e901392e21abc5cef690ec9 WatchSource:0}: Error finding container 18ec664fa94a5f0ad6619650b06e1aa23f4cad594e901392e21abc5cef690ec9: Status 404 returned error can't find the container with id 18ec664fa94a5f0ad6619650b06e1aa23f4cad594e901392e21abc5cef690ec9 Jul 28 12:14:21.348632 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:21.348593 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" event={"ID":"354656be-9109-4915-873f-f945e8959c33","Type":"ContainerStarted","Data":"43d5d694864654331db120d143951d675623e66400db54474aebec9e2df3a024"} Jul 28 12:14:21.348824 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:21.348647 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" event={"ID":"354656be-9109-4915-873f-f945e8959c33","Type":"ContainerStarted","Data":"18ec664fa94a5f0ad6619650b06e1aa23f4cad594e901392e21abc5cef690ec9"} Jul 28 12:14:21.351804 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:21.351772 2575 generic.go:356] "Generic (PLEG): container finished" podID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerID="6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2" exitCode=0 Jul 28 12:14:21.351914 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:21.351814 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" event={"ID":"149d6ee4-fa3d-498a-9ff5-9870f83f12ea","Type":"ContainerDied","Data":"6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2"} Jul 28 12:14:26.381147 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:26.381113 2575 generic.go:356] "Generic (PLEG): container finished" podID="354656be-9109-4915-873f-f945e8959c33" containerID="43d5d694864654331db120d143951d675623e66400db54474aebec9e2df3a024" exitCode=0 Jul 28 12:14:26.381147 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:26.381150 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" event={"ID":"354656be-9109-4915-873f-f945e8959c33","Type":"ContainerDied","Data":"43d5d694864654331db120d143951d675623e66400db54474aebec9e2df3a024"} Jul 28 12:14:27.321358 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:27.321296 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused" Jul 28 12:14:27.386886 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:27.386856 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" event={"ID":"354656be-9109-4915-873f-f945e8959c33","Type":"ContainerStarted","Data":"256945881b6987633a0eb02f26a409feac3d913d7a0aa91b12a97d2a8f662649"} Jul 28 12:14:28.088983 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:28.088941 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Jul 28 12:14:32.408166 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:32.408077 2575 generic.go:356] "Generic (PLEG): container finished" podID="354656be-9109-4915-873f-f945e8959c33" containerID="256945881b6987633a0eb02f26a409feac3d913d7a0aa91b12a97d2a8f662649" exitCode=0 Jul 28 12:14:32.408166 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:32.408152 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" event={"ID":"354656be-9109-4915-873f-f945e8959c33","Type":"ContainerDied","Data":"256945881b6987633a0eb02f26a409feac3d913d7a0aa91b12a97d2a8f662649"} Jul 28 12:14:33.419010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:33.418970 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" event={"ID":"354656be-9109-4915-873f-f945e8959c33","Type":"ContainerStarted","Data":"3629434f1040f2c65e8fcc0080ce81c8154dd7ccff75d3e1ce05c2d89367bb84"} Jul 28 12:14:33.419532 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:33.419032 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" event={"ID":"354656be-9109-4915-873f-f945e8959c33","Type":"ContainerStarted","Data":"2589023fc4abed90f1a857b09be0b51ead3f035c5717daed493e2922cf530ce9"} Jul 28 12:14:33.419532 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:33.419051 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" event={"ID":"354656be-9109-4915-873f-f945e8959c33","Type":"ContainerStarted","Data":"9f88b0c972dac6459c7dde91dba558e76b6d7d12557712d9a86e13e73adbc544"} Jul 28 12:14:33.419532 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:33.419071 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" event={"ID":"354656be-9109-4915-873f-f945e8959c33","Type":"ContainerStarted","Data":"dd8e257556c0ad5970d63343b57dfa0c130f0a1b7537841fd41c75e57860f506"} Jul 28 12:14:33.459883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:33.459810 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" podStartSLOduration=14.459749831 podStartE2EDuration="14.459749831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:14:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:14:33.457854162 +0000 UTC m=+507.782519451" watchObservedRunningTime="2026-07-28 12:14:33.459749831 +0000 UTC m=+507.784415111" Jul 28 12:14:35.858473 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:35.858438 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:35.859019 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:35.858995 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:35.859176 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:35.859026 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:35.859176 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:35.859046 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:35.860636 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:35.860607 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.28:6571: connect: connection refused" Jul 28 12:14:35.860830 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:35.860632 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Jul 28 12:14:35.860922 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:35.860670 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.28:8816: connect: connection refused" Jul 28 12:14:35.860922 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:35.860665 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Jul 28 12:14:37.321683 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:37.321633 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused" Jul 28 12:14:38.089562 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:38.089514 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Jul 28 12:14:38.858455 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:38.858420 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:38.858898 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:38.858632 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:38.859039 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:38.859010 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:38.859168 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:38.859054 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:38.859168 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:38.859073 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:38.859168 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:38.859090 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:38.859462 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:38.859445 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:38.859521 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:38.859498 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:38.859555 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:38.859530 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:38.863473 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:38.863456 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:39.452539 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:39.452504 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:39.455651 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:39.455629 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:14:47.321457 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:47.321406 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.27:8443: connect: connection refused" Jul 28 12:14:47.321913 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:47.321554 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:14:48.089192 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:48.089149 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="online" probeResult="failure" output="Get \"https://10.133.0.27:6567/health\": dial tcp 10.133.0.27:6567: connect: connection refused" Jul 28 12:14:48.089387 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:48.089270 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:14:50.320842 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.320818 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:14:50.369006 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.368974 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-offline-tls\") pod \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " Jul 28 12:14:50.369006 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.369005 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-ui-tls\") pod \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " Jul 28 12:14:50.369244 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.369034 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-registry-tls\") pod \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " Jul 28 12:14:50.369244 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.369161 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7v64r\" (UniqueName: \"kubernetes.io/projected/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-kube-api-access-7v64r\") pod \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " Jul 28 12:14:50.369383 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.369253 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-feast-data\") pod \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " Jul 28 12:14:50.369383 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.369284 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-online-tls\") pod \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\" (UID: \"149d6ee4-fa3d-498a-9ff5-9870f83f12ea\") " Jul 28 12:14:50.369945 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.369899 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-feast-data" (OuterVolumeSpecName: "feast-data") pod "149d6ee4-fa3d-498a-9ff5-9870f83f12ea" (UID: "149d6ee4-fa3d-498a-9ff5-9870f83f12ea"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 12:14:50.371592 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.371562 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "149d6ee4-fa3d-498a-9ff5-9870f83f12ea" (UID: "149d6ee4-fa3d-498a-9ff5-9870f83f12ea"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:14:50.371700 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.371634 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "149d6ee4-fa3d-498a-9ff5-9870f83f12ea" (UID: "149d6ee4-fa3d-498a-9ff5-9870f83f12ea"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:14:50.371700 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.371656 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "149d6ee4-fa3d-498a-9ff5-9870f83f12ea" (UID: "149d6ee4-fa3d-498a-9ff5-9870f83f12ea"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:14:50.371800 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.371717 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-online-tls" (OuterVolumeSpecName: "online-tls") pod "149d6ee4-fa3d-498a-9ff5-9870f83f12ea" (UID: "149d6ee4-fa3d-498a-9ff5-9870f83f12ea"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:14:50.371800 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.371746 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-kube-api-access-7v64r" (OuterVolumeSpecName: "kube-api-access-7v64r") pod "149d6ee4-fa3d-498a-9ff5-9870f83f12ea" (UID: "149d6ee4-fa3d-498a-9ff5-9870f83f12ea"). InnerVolumeSpecName "kube-api-access-7v64r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 12:14:50.470410 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.470344 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7v64r\" (UniqueName: \"kubernetes.io/projected/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-kube-api-access-7v64r\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:14:50.470410 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.470365 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-feast-data\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:14:50.470410 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.470395 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-online-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:14:50.470410 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.470403 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-offline-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:14:50.470410 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.470411 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-ui-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:14:50.470627 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.470418 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/149d6ee4-fa3d-498a-9ff5-9870f83f12ea-registry-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:14:50.497360 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.497334 2575 generic.go:356] "Generic (PLEG): container finished" podID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerID="f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2" exitCode=137 Jul 28 12:14:50.497360 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.497359 2575 generic.go:356] "Generic (PLEG): container finished" podID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerID="749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b" exitCode=137 Jul 28 12:14:50.497527 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.497395 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" Jul 28 12:14:50.497527 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.497418 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" event={"ID":"149d6ee4-fa3d-498a-9ff5-9870f83f12ea","Type":"ContainerDied","Data":"f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2"} Jul 28 12:14:50.497527 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.497468 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" event={"ID":"149d6ee4-fa3d-498a-9ff5-9870f83f12ea","Type":"ContainerDied","Data":"749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b"} Jul 28 12:14:50.497527 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.497484 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr" event={"ID":"149d6ee4-fa3d-498a-9ff5-9870f83f12ea","Type":"ContainerDied","Data":"1ab0993d44d15d334e99bd5a737533c8c9ad8ed747db8dff7fc328995102e5f8"} Jul 28 12:14:50.497527 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.497501 2575 scope.go:117] "RemoveContainer" containerID="0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f" Jul 28 12:14:50.506007 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.505989 2575 scope.go:117] "RemoveContainer" containerID="f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2" Jul 28 12:14:50.513001 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.512988 2575 scope.go:117] "RemoveContainer" containerID="6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2" Jul 28 12:14:50.519820 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.519805 2575 scope.go:117] "RemoveContainer" containerID="749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b" Jul 28 12:14:50.524188 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.524167 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr"] Jul 28 12:14:50.526982 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.526966 2575 scope.go:117] "RemoveContainer" containerID="a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3" Jul 28 12:14:50.534239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.534193 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c756fdd-kvbwr"] Jul 28 12:14:50.536064 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.536046 2575 scope.go:117] "RemoveContainer" containerID="3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef" Jul 28 12:14:50.542964 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.542939 2575 scope.go:117] "RemoveContainer" containerID="0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f" Jul 28 12:14:50.543196 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:14:50.543177 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f\": container with ID starting with 0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f not found: ID does not exist" containerID="0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f" Jul 28 12:14:50.543255 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.543204 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f"} err="failed to get container status \"0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f\": rpc error: code = NotFound desc = could not find container \"0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f\": container with ID starting with 0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f not found: ID does not exist" Jul 28 12:14:50.543255 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.543223 2575 scope.go:117] "RemoveContainer" containerID="f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2" Jul 28 12:14:50.543478 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:14:50.543461 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2\": container with ID starting with f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2 not found: ID does not exist" containerID="f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2" Jul 28 12:14:50.543520 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.543482 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2"} err="failed to get container status \"f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2\": rpc error: code = NotFound desc = could not find container \"f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2\": container with ID starting with f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2 not found: ID does not exist" Jul 28 12:14:50.543520 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.543496 2575 scope.go:117] "RemoveContainer" containerID="6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2" Jul 28 12:14:50.543725 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:14:50.543709 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2\": container with ID starting with 6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2 not found: ID does not exist" containerID="6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2" Jul 28 12:14:50.543766 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.543730 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2"} err="failed to get container status \"6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2\": rpc error: code = NotFound desc = could not find container \"6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2\": container with ID starting with 6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2 not found: ID does not exist" Jul 28 12:14:50.543766 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.543745 2575 scope.go:117] "RemoveContainer" containerID="749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b" Jul 28 12:14:50.543975 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:14:50.543958 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b\": container with ID starting with 749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b not found: ID does not exist" containerID="749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b" Jul 28 12:14:50.544025 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.543978 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b"} err="failed to get container status \"749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b\": rpc error: code = NotFound desc = could not find container \"749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b\": container with ID starting with 749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b not found: ID does not exist" Jul 28 12:14:50.544025 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.543991 2575 scope.go:117] "RemoveContainer" containerID="a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3" Jul 28 12:14:50.544229 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:14:50.544212 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3\": container with ID starting with a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3 not found: ID does not exist" containerID="a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3" Jul 28 12:14:50.544274 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.544240 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3"} err="failed to get container status \"a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3\": rpc error: code = NotFound desc = could not find container \"a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3\": container with ID starting with a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3 not found: ID does not exist" Jul 28 12:14:50.544274 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.544255 2575 scope.go:117] "RemoveContainer" containerID="3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef" Jul 28 12:14:50.544478 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:14:50.544461 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef\": container with ID starting with 3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef not found: ID does not exist" containerID="3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef" Jul 28 12:14:50.544515 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.544483 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef"} err="failed to get container status \"3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef\": rpc error: code = NotFound desc = could not find container \"3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef\": container with ID starting with 3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef not found: ID does not exist" Jul 28 12:14:50.544515 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.544495 2575 scope.go:117] "RemoveContainer" containerID="0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f" Jul 28 12:14:50.544721 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.544704 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f"} err="failed to get container status \"0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f\": rpc error: code = NotFound desc = could not find container \"0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f\": container with ID starting with 0a4ab458e08f2b6442ba607c62bb9aa2a1f2e697020f130a3a96060c308bb34f not found: ID does not exist" Jul 28 12:14:50.544770 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.544722 2575 scope.go:117] "RemoveContainer" containerID="f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2" Jul 28 12:14:50.544923 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.544907 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2"} err="failed to get container status \"f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2\": rpc error: code = NotFound desc = could not find container \"f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2\": container with ID starting with f78ef722ccc03a2a782847eecf83ddb7f443311ee26e0573f6a13d6a2c6ef5c2 not found: ID does not exist" Jul 28 12:14:50.544965 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.544924 2575 scope.go:117] "RemoveContainer" containerID="6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2" Jul 28 12:14:50.545094 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.545079 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2"} err="failed to get container status \"6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2\": rpc error: code = NotFound desc = could not find container \"6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2\": container with ID starting with 6ed7130f9b1967c1fbb64cdaf56dd808e4fbaea02e8e514ef627553932f52de2 not found: ID does not exist" Jul 28 12:14:50.545094 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.545094 2575 scope.go:117] "RemoveContainer" containerID="749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b" Jul 28 12:14:50.545259 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.545242 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b"} err="failed to get container status \"749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b\": rpc error: code = NotFound desc = could not find container \"749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b\": container with ID starting with 749da796390af2c8901907f70ae8b92ebd68e15478bdfa836a3d8bd74827960b not found: ID does not exist" Jul 28 12:14:50.545322 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.545259 2575 scope.go:117] "RemoveContainer" containerID="a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3" Jul 28 12:14:50.545493 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.545474 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3"} err="failed to get container status \"a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3\": rpc error: code = NotFound desc = could not find container \"a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3\": container with ID starting with a2cb9187ad99c646d0bfb105b0a04d7b34fafd0cb608ac38660eeb7a657646a3 not found: ID does not exist" Jul 28 12:14:50.545493 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.545493 2575 scope.go:117] "RemoveContainer" containerID="3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef" Jul 28 12:14:50.545714 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:50.545697 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef"} err="failed to get container status \"3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef\": rpc error: code = NotFound desc = could not find container \"3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef\": container with ID starting with 3a445963adedc9b3450a3fb0e3040ea6832e4135785c544c1efa1d59758247ef not found: ID does not exist" Jul 28 12:14:52.209213 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:14:52.209182 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" path="/var/lib/kubelet/pods/149d6ee4-fa3d-498a-9ff5-9870f83f12ea/volumes" Jul 28 12:15:41.871885 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.871851 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7"] Jul 28 12:15:41.872353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.872141 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="registry" Jul 28 12:15:41.872353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.872152 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="registry" Jul 28 12:15:41.872353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.872165 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="offline" Jul 28 12:15:41.872353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.872170 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="offline" Jul 28 12:15:41.872353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.872178 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="online" Jul 28 12:15:41.872353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.872184 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="online" Jul 28 12:15:41.872353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.872192 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="feast-apply" Jul 28 12:15:41.872353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.872197 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="feast-apply" Jul 28 12:15:41.872353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.872204 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="ui" Jul 28 12:15:41.872353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.872209 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="ui" Jul 28 12:15:41.872353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.872218 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="feast-init" Jul 28 12:15:41.872353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.872224 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="feast-init" Jul 28 12:15:41.872353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.872269 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="online" Jul 28 12:15:41.872353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.872277 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="registry" Jul 28 12:15:41.872353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.872283 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="offline" Jul 28 12:15:41.872353 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.872289 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="149d6ee4-fa3d-498a-9ff5-9870f83f12ea" containerName="ui" Jul 28 12:15:41.875707 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.875691 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:41.878608 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.878588 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-25vpv\"" Jul 28 12:15:41.880393 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.880377 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jul 28 12:15:41.880393 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.880383 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jul 28 12:15:41.880500 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.880401 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Jul 28 12:15:41.881639 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.881616 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jul 28 12:15:41.881765 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.881746 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jul 28 12:15:41.881825 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.881799 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jul 28 12:15:41.882215 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.881959 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jul 28 12:15:41.901124 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.901101 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7"] Jul 28 12:15:41.949771 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.949752 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-offline-tls\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:41.949876 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.949784 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl7bp\" (UniqueName: \"kubernetes.io/projected/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-kube-api-access-sl7bp\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:41.949876 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.949808 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-feast-data\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:41.949876 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.949829 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-online-tls\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:41.949993 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.949916 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-registry-tls\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:41.949993 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:41.949970 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-ui-tls\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:42.050619 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:42.050596 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl7bp\" (UniqueName: \"kubernetes.io/projected/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-kube-api-access-sl7bp\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:42.050734 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:42.050644 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-feast-data\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:42.050734 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:42.050680 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-online-tls\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:42.050734 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:42.050709 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-registry-tls\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:42.050896 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:42.050773 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-ui-tls\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:42.050896 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:42.050820 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-offline-tls\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:42.050998 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:15:42.050921 2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Jul 28 12:15:42.051052 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:15:42.051000 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-ui-tls podName:f7fc7aff-a8cf-4632-a4f7-517dc7b4db42 nodeName:}" failed. No retries permitted until 2026-07-28 12:15:42.550980204 +0000 UTC m=+576.875645465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-ui-tls") pod "feast-simple-feast-remote-setup-5795d94dd9-v4dq7" (UID: "f7fc7aff-a8cf-4632-a4f7-517dc7b4db42") : secret "feast-simple-feast-remote-setup-ui-tls" not found Jul 28 12:15:42.051052 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:15:42.050925 2575 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found Jul 28 12:15:42.051157 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:15:42.051082 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-offline-tls podName:f7fc7aff-a8cf-4632-a4f7-517dc7b4db42 nodeName:}" failed. No retries permitted until 2026-07-28 12:15:42.551063825 +0000 UTC m=+576.875729082 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-offline-tls") pod "feast-simple-feast-remote-setup-5795d94dd9-v4dq7" (UID: "f7fc7aff-a8cf-4632-a4f7-517dc7b4db42") : secret "feast-simple-feast-remote-setup-offline-tls" not found Jul 28 12:15:42.051157 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:42.051085 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-feast-data\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:42.051383 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:42.051367 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-registry-tls\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:42.053234 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:42.053211 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-online-tls\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:42.059771 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:42.059752 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl7bp\" (UniqueName: \"kubernetes.io/projected/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-kube-api-access-sl7bp\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:42.554818 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:42.554789 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-ui-tls\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:42.554979 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:42.554832 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-offline-tls\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:42.557114 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:42.557081 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-ui-tls\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:42.557229 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:42.557133 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-offline-tls\") pod \"feast-simple-feast-remote-setup-5795d94dd9-v4dq7\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:42.784871 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:42.784840 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:42.921110 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:42.921088 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7"] Jul 28 12:15:42.922292 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:15:42.922272 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7fc7aff_a8cf_4632_a4f7_517dc7b4db42.slice/crio-3de975ca571bcfdb81823013665e858a57fbcd988290205be641b692e778c16c WatchSource:0}: Error finding container 3de975ca571bcfdb81823013665e858a57fbcd988290205be641b692e778c16c: Status 404 returned error can't find the container with id 3de975ca571bcfdb81823013665e858a57fbcd988290205be641b692e778c16c Jul 28 12:15:43.703770 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:43.703728 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" event={"ID":"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42","Type":"ContainerStarted","Data":"926e8b65e1527c075f7eb99207653c823eaecf021fb416d780174082d0807fcc"} Jul 28 12:15:43.703931 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:43.703787 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" event={"ID":"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42","Type":"ContainerStarted","Data":"3de975ca571bcfdb81823013665e858a57fbcd988290205be641b692e778c16c"} Jul 28 12:15:47.721947 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:47.721920 2575 generic.go:356] "Generic (PLEG): container finished" podID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerID="926e8b65e1527c075f7eb99207653c823eaecf021fb416d780174082d0807fcc" exitCode=0 Jul 28 12:15:47.722394 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:47.721996 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" event={"ID":"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42","Type":"ContainerDied","Data":"926e8b65e1527c075f7eb99207653c823eaecf021fb416d780174082d0807fcc"} Jul 28 12:15:48.728118 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:48.728087 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" event={"ID":"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42","Type":"ContainerStarted","Data":"9368e31edb931eda372052fcc637f2f100b32738dbc7861b1039cd8d614d6134"} Jul 28 12:15:53.753787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:53.753712 2575 generic.go:356] "Generic (PLEG): container finished" podID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerID="9368e31edb931eda372052fcc637f2f100b32738dbc7861b1039cd8d614d6134" exitCode=0 Jul 28 12:15:53.753787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:53.753756 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" event={"ID":"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42","Type":"ContainerDied","Data":"9368e31edb931eda372052fcc637f2f100b32738dbc7861b1039cd8d614d6134"} Jul 28 12:15:54.762847 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:54.762802 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" event={"ID":"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42","Type":"ContainerStarted","Data":"c2cb929eda62fc0cab6e4d13040e0fdefb7df453bbbdee11751b371e23ff3e90"} Jul 28 12:15:54.763355 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:54.762867 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" event={"ID":"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42","Type":"ContainerStarted","Data":"018f306de7a009d2922f3f8984a0d2be7cb160e81f93d043e592c332fcf34e3c"} Jul 28 12:15:54.763355 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:54.762888 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" event={"ID":"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42","Type":"ContainerStarted","Data":"61460392df58122581924a48cf5c8e8fce157cebe3fd60b7a5baf1aab2f26049"} Jul 28 12:15:54.785296 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:54.785253 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:54.785464 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:54.785327 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:54.785464 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:54.785350 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:15:54.787018 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:54.786986 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Jul 28 12:15:54.787018 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:54.787005 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Jul 28 12:15:54.787225 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:54.787009 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.29:8816: connect: connection refused" Jul 28 12:15:54.795915 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:54.795857 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" podStartSLOduration=13.795837597 podStartE2EDuration="13.795837597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:15:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:15:54.792747423 +0000 UTC m=+589.117412704" watchObservedRunningTime="2026-07-28 12:15:54.795837597 +0000 UTC m=+589.120502917" Jul 28 12:15:57.785177 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:57.785118 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Jul 28 12:15:57.785177 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:57.785146 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.29:8816: connect: connection refused" Jul 28 12:15:57.785750 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:15:57.785257 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Jul 28 12:16:00.786696 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:16:00.786665 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:16:00.787085 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:16:00.786922 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:16:00.787232 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:16:00.787211 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:16:00.787359 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:16:00.787255 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:16:00.787489 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:16:00.787465 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:16:00.787691 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:16:00.787677 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:16:00.793225 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:16:00.793187 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:16:01.794898 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:16:01.794871 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:16:01.799262 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:16:01.799237 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:16:06.093983 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:16:06.093958 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/1.log" Jul 28 12:16:06.098185 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:16:06.098165 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/1.log" Jul 28 12:17:03.398657 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:03.398624 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7"] Jul 28 12:17:03.399155 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:03.398932 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="online" containerID="cri-o://61460392df58122581924a48cf5c8e8fce157cebe3fd60b7a5baf1aab2f26049" gracePeriod=30 Jul 28 12:17:03.399155 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:03.398996 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="ui" containerID="cri-o://c2cb929eda62fc0cab6e4d13040e0fdefb7df453bbbdee11751b371e23ff3e90" gracePeriod=30 Jul 28 12:17:03.399155 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:03.399003 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="offline" containerID="cri-o://018f306de7a009d2922f3f8984a0d2be7cb160e81f93d043e592c332fcf34e3c" gracePeriod=30 Jul 28 12:17:03.537404 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:03.537372 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p"] Jul 28 12:17:03.537706 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:03.537673 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="ui" containerID="cri-o://3629434f1040f2c65e8fcc0080ce81c8154dd7ccff75d3e1ce05c2d89367bb84" gracePeriod=30 Jul 28 12:17:03.537942 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:03.537668 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="registry" containerID="cri-o://dd8e257556c0ad5970d63343b57dfa0c130f0a1b7537841fd41c75e57860f506" gracePeriod=30 Jul 28 12:17:03.538051 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:03.537716 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="online" containerID="cri-o://9f88b0c972dac6459c7dde91dba558e76b6d7d12557712d9a86e13e73adbc544" gracePeriod=30 Jul 28 12:17:03.538171 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:03.537738 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="offline" containerID="cri-o://2589023fc4abed90f1a857b09be0b51ead3f035c5717daed493e2922cf530ce9" gracePeriod=30 Jul 28 12:17:04.047794 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:04.047766 2575 generic.go:356] "Generic (PLEG): container finished" podID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerID="c2cb929eda62fc0cab6e4d13040e0fdefb7df453bbbdee11751b371e23ff3e90" exitCode=0 Jul 28 12:17:04.047911 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:04.047823 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" event={"ID":"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42","Type":"ContainerDied","Data":"c2cb929eda62fc0cab6e4d13040e0fdefb7df453bbbdee11751b371e23ff3e90"} Jul 28 12:17:05.053536 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:05.053502 2575 generic.go:356] "Generic (PLEG): container finished" podID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerID="61460392df58122581924a48cf5c8e8fce157cebe3fd60b7a5baf1aab2f26049" exitCode=0 Jul 28 12:17:05.053893 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:05.053570 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" event={"ID":"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42","Type":"ContainerDied","Data":"61460392df58122581924a48cf5c8e8fce157cebe3fd60b7a5baf1aab2f26049"} Jul 28 12:17:05.056267 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:05.056246 2575 generic.go:356] "Generic (PLEG): container finished" podID="354656be-9109-4915-873f-f945e8959c33" containerID="3629434f1040f2c65e8fcc0080ce81c8154dd7ccff75d3e1ce05c2d89367bb84" exitCode=0 Jul 28 12:17:05.056267 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:05.056263 2575 generic.go:356] "Generic (PLEG): container finished" podID="354656be-9109-4915-873f-f945e8959c33" containerID="9f88b0c972dac6459c7dde91dba558e76b6d7d12557712d9a86e13e73adbc544" exitCode=0 Jul 28 12:17:05.056420 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:05.056279 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" event={"ID":"354656be-9109-4915-873f-f945e8959c33","Type":"ContainerDied","Data":"3629434f1040f2c65e8fcc0080ce81c8154dd7ccff75d3e1ce05c2d89367bb84"} Jul 28 12:17:05.056420 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:05.056300 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" event={"ID":"354656be-9109-4915-873f-f945e8959c33","Type":"ContainerDied","Data":"9f88b0c972dac6459c7dde91dba558e76b6d7d12557712d9a86e13e73adbc544"} Jul 28 12:17:08.859542 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:08.859497 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Jul 28 12:17:09.453071 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:09.453026 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Jul 28 12:17:10.788001 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:10.787957 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Jul 28 12:17:11.795222 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:11.795181 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Jul 28 12:17:18.859361 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:18.859290 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Jul 28 12:17:19.452673 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:19.452630 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Jul 28 12:17:20.787714 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:20.787669 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Jul 28 12:17:21.795124 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:21.795082 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Jul 28 12:17:28.859553 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:28.859508 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.28:8443: connect: connection refused" Jul 28 12:17:28.859977 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:28.859648 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:17:29.453658 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:29.453616 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="online" probeResult="failure" output="Get \"https://10.133.0.28:6567/health\": dial tcp 10.133.0.28:6567: connect: connection refused" Jul 28 12:17:29.453862 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:29.453761 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:17:30.788299 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:30.788247 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="ui" probeResult="failure" output="dial tcp 10.133.0.29:8443: connect: connection refused" Jul 28 12:17:30.788718 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:30.788413 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:17:31.795347 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:31.795285 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="online" probeResult="failure" output="Get \"https://10.133.0.29:6567/health\": dial tcp 10.133.0.29:6567: connect: connection refused" Jul 28 12:17:31.795722 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:31.795442 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:17:34.048234 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.048212 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:17:34.143523 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.143496 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sl7bp\" (UniqueName: \"kubernetes.io/projected/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-kube-api-access-sl7bp\") pod \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " Jul 28 12:17:34.143672 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.143533 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-ui-tls\") pod \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " Jul 28 12:17:34.143672 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.143571 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-offline-tls\") pod \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " Jul 28 12:17:34.143672 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.143595 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-registry-tls\") pod \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " Jul 28 12:17:34.143672 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.143624 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-feast-data\") pod \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " Jul 28 12:17:34.143672 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.143663 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-online-tls\") pod \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\" (UID: \"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42\") " Jul 28 12:17:34.144018 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.143994 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" (UID: "f7fc7aff-a8cf-4632-a4f7-517dc7b4db42"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 12:17:34.144244 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.144200 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-feast-data" (OuterVolumeSpecName: "feast-data") pod "f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" (UID: "f7fc7aff-a8cf-4632-a4f7-517dc7b4db42"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 12:17:34.146597 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.146571 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-kube-api-access-sl7bp" (OuterVolumeSpecName: "kube-api-access-sl7bp") pod "f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" (UID: "f7fc7aff-a8cf-4632-a4f7-517dc7b4db42"). InnerVolumeSpecName "kube-api-access-sl7bp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 12:17:34.148601 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.148572 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-online-tls" (OuterVolumeSpecName: "online-tls") pod "f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" (UID: "f7fc7aff-a8cf-4632-a4f7-517dc7b4db42"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:17:34.148681 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.148633 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" (UID: "f7fc7aff-a8cf-4632-a4f7-517dc7b4db42"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:17:34.148681 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.148647 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" (UID: "f7fc7aff-a8cf-4632-a4f7-517dc7b4db42"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:17:34.174600 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.174574 2575 generic.go:356] "Generic (PLEG): container finished" podID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerID="018f306de7a009d2922f3f8984a0d2be7cb160e81f93d043e592c332fcf34e3c" exitCode=137 Jul 28 12:17:34.174706 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.174638 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" Jul 28 12:17:34.174706 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.174653 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" event={"ID":"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42","Type":"ContainerDied","Data":"018f306de7a009d2922f3f8984a0d2be7cb160e81f93d043e592c332fcf34e3c"} Jul 28 12:17:34.174786 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.174710 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7" event={"ID":"f7fc7aff-a8cf-4632-a4f7-517dc7b4db42","Type":"ContainerDied","Data":"3de975ca571bcfdb81823013665e858a57fbcd988290205be641b692e778c16c"} Jul 28 12:17:34.174786 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.174736 2575 scope.go:117] "RemoveContainer" containerID="c2cb929eda62fc0cab6e4d13040e0fdefb7df453bbbdee11751b371e23ff3e90" Jul 28 12:17:34.178088 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.178066 2575 generic.go:356] "Generic (PLEG): container finished" podID="354656be-9109-4915-873f-f945e8959c33" containerID="2589023fc4abed90f1a857b09be0b51ead3f035c5717daed493e2922cf530ce9" exitCode=137 Jul 28 12:17:34.178088 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.178089 2575 generic.go:356] "Generic (PLEG): container finished" podID="354656be-9109-4915-873f-f945e8959c33" containerID="dd8e257556c0ad5970d63343b57dfa0c130f0a1b7537841fd41c75e57860f506" exitCode=137 Jul 28 12:17:34.178247 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.178112 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" event={"ID":"354656be-9109-4915-873f-f945e8959c33","Type":"ContainerDied","Data":"2589023fc4abed90f1a857b09be0b51ead3f035c5717daed493e2922cf530ce9"} Jul 28 12:17:34.178247 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.178140 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" event={"ID":"354656be-9109-4915-873f-f945e8959c33","Type":"ContainerDied","Data":"dd8e257556c0ad5970d63343b57dfa0c130f0a1b7537841fd41c75e57860f506"} Jul 28 12:17:34.182729 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.182615 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:17:34.184044 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.184031 2575 scope.go:117] "RemoveContainer" containerID="018f306de7a009d2922f3f8984a0d2be7cb160e81f93d043e592c332fcf34e3c" Jul 28 12:17:34.190787 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.190772 2575 scope.go:117] "RemoveContainer" containerID="61460392df58122581924a48cf5c8e8fce157cebe3fd60b7a5baf1aab2f26049" Jul 28 12:17:34.198166 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.198149 2575 scope.go:117] "RemoveContainer" containerID="9368e31edb931eda372052fcc637f2f100b32738dbc7861b1039cd8d614d6134" Jul 28 12:17:34.198859 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.198842 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7"] Jul 28 12:17:34.202437 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.202416 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5795d94dd9-v4dq7"] Jul 28 12:17:34.205918 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.205905 2575 scope.go:117] "RemoveContainer" containerID="926e8b65e1527c075f7eb99207653c823eaecf021fb416d780174082d0807fcc" Jul 28 12:17:34.208564 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.208539 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" path="/var/lib/kubelet/pods/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42/volumes" Jul 28 12:17:34.213590 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.213577 2575 scope.go:117] "RemoveContainer" containerID="c2cb929eda62fc0cab6e4d13040e0fdefb7df453bbbdee11751b371e23ff3e90" Jul 28 12:17:34.213808 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:17:34.213790 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2cb929eda62fc0cab6e4d13040e0fdefb7df453bbbdee11751b371e23ff3e90\": container with ID starting with c2cb929eda62fc0cab6e4d13040e0fdefb7df453bbbdee11751b371e23ff3e90 not found: ID does not exist" containerID="c2cb929eda62fc0cab6e4d13040e0fdefb7df453bbbdee11751b371e23ff3e90" Jul 28 12:17:34.213880 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.213821 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2cb929eda62fc0cab6e4d13040e0fdefb7df453bbbdee11751b371e23ff3e90"} err="failed to get container status \"c2cb929eda62fc0cab6e4d13040e0fdefb7df453bbbdee11751b371e23ff3e90\": rpc error: code = NotFound desc = could not find container \"c2cb929eda62fc0cab6e4d13040e0fdefb7df453bbbdee11751b371e23ff3e90\": container with ID starting with c2cb929eda62fc0cab6e4d13040e0fdefb7df453bbbdee11751b371e23ff3e90 not found: ID does not exist" Jul 28 12:17:34.213880 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.213846 2575 scope.go:117] "RemoveContainer" containerID="018f306de7a009d2922f3f8984a0d2be7cb160e81f93d043e592c332fcf34e3c" Jul 28 12:17:34.214078 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:17:34.214062 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"018f306de7a009d2922f3f8984a0d2be7cb160e81f93d043e592c332fcf34e3c\": container with ID starting with 018f306de7a009d2922f3f8984a0d2be7cb160e81f93d043e592c332fcf34e3c not found: ID does not exist" containerID="018f306de7a009d2922f3f8984a0d2be7cb160e81f93d043e592c332fcf34e3c" Jul 28 12:17:34.214161 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.214085 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"018f306de7a009d2922f3f8984a0d2be7cb160e81f93d043e592c332fcf34e3c"} err="failed to get container status \"018f306de7a009d2922f3f8984a0d2be7cb160e81f93d043e592c332fcf34e3c\": rpc error: code = NotFound desc = could not find container \"018f306de7a009d2922f3f8984a0d2be7cb160e81f93d043e592c332fcf34e3c\": container with ID starting with 018f306de7a009d2922f3f8984a0d2be7cb160e81f93d043e592c332fcf34e3c not found: ID does not exist" Jul 28 12:17:34.214161 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.214100 2575 scope.go:117] "RemoveContainer" containerID="61460392df58122581924a48cf5c8e8fce157cebe3fd60b7a5baf1aab2f26049" Jul 28 12:17:34.214382 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:17:34.214366 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61460392df58122581924a48cf5c8e8fce157cebe3fd60b7a5baf1aab2f26049\": container with ID starting with 61460392df58122581924a48cf5c8e8fce157cebe3fd60b7a5baf1aab2f26049 not found: ID does not exist" containerID="61460392df58122581924a48cf5c8e8fce157cebe3fd60b7a5baf1aab2f26049" Jul 28 12:17:34.214444 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.214389 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61460392df58122581924a48cf5c8e8fce157cebe3fd60b7a5baf1aab2f26049"} err="failed to get container status \"61460392df58122581924a48cf5c8e8fce157cebe3fd60b7a5baf1aab2f26049\": rpc error: code = NotFound desc = could not find container \"61460392df58122581924a48cf5c8e8fce157cebe3fd60b7a5baf1aab2f26049\": container with ID starting with 61460392df58122581924a48cf5c8e8fce157cebe3fd60b7a5baf1aab2f26049 not found: ID does not exist" Jul 28 12:17:34.214444 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.214404 2575 scope.go:117] "RemoveContainer" containerID="9368e31edb931eda372052fcc637f2f100b32738dbc7861b1039cd8d614d6134" Jul 28 12:17:34.214623 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:17:34.214606 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9368e31edb931eda372052fcc637f2f100b32738dbc7861b1039cd8d614d6134\": container with ID starting with 9368e31edb931eda372052fcc637f2f100b32738dbc7861b1039cd8d614d6134 not found: ID does not exist" containerID="9368e31edb931eda372052fcc637f2f100b32738dbc7861b1039cd8d614d6134" Jul 28 12:17:34.214663 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.214627 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9368e31edb931eda372052fcc637f2f100b32738dbc7861b1039cd8d614d6134"} err="failed to get container status \"9368e31edb931eda372052fcc637f2f100b32738dbc7861b1039cd8d614d6134\": rpc error: code = NotFound desc = could not find container \"9368e31edb931eda372052fcc637f2f100b32738dbc7861b1039cd8d614d6134\": container with ID starting with 9368e31edb931eda372052fcc637f2f100b32738dbc7861b1039cd8d614d6134 not found: ID does not exist" Jul 28 12:17:34.214663 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.214641 2575 scope.go:117] "RemoveContainer" containerID="926e8b65e1527c075f7eb99207653c823eaecf021fb416d780174082d0807fcc" Jul 28 12:17:34.214857 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:17:34.214841 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"926e8b65e1527c075f7eb99207653c823eaecf021fb416d780174082d0807fcc\": container with ID starting with 926e8b65e1527c075f7eb99207653c823eaecf021fb416d780174082d0807fcc not found: ID does not exist" containerID="926e8b65e1527c075f7eb99207653c823eaecf021fb416d780174082d0807fcc" Jul 28 12:17:34.214903 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.214859 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"926e8b65e1527c075f7eb99207653c823eaecf021fb416d780174082d0807fcc"} err="failed to get container status \"926e8b65e1527c075f7eb99207653c823eaecf021fb416d780174082d0807fcc\": rpc error: code = NotFound desc = could not find container \"926e8b65e1527c075f7eb99207653c823eaecf021fb416d780174082d0807fcc\": container with ID starting with 926e8b65e1527c075f7eb99207653c823eaecf021fb416d780174082d0807fcc not found: ID does not exist" Jul 28 12:17:34.244486 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.244467 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-online-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:17:34.244486 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.244487 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sl7bp\" (UniqueName: \"kubernetes.io/projected/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-kube-api-access-sl7bp\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:17:34.244602 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.244496 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-ui-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:17:34.244602 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.244507 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-offline-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:17:34.244602 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.244515 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-registry-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:17:34.244602 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.244522 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f7fc7aff-a8cf-4632-a4f7-517dc7b4db42-feast-data\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:17:34.345729 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.345701 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/354656be-9109-4915-873f-f945e8959c33-feast-data\") pod \"354656be-9109-4915-873f-f945e8959c33\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " Jul 28 12:17:34.345859 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.345746 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-registry-tls\") pod \"354656be-9109-4915-873f-f945e8959c33\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " Jul 28 12:17:34.345859 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.345769 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-offline-tls\") pod \"354656be-9109-4915-873f-f945e8959c33\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " Jul 28 12:17:34.345859 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.345790 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnp94\" (UniqueName: \"kubernetes.io/projected/354656be-9109-4915-873f-f945e8959c33-kube-api-access-nnp94\") pod \"354656be-9109-4915-873f-f945e8959c33\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " Jul 28 12:17:34.345859 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.345838 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-ui-tls\") pod \"354656be-9109-4915-873f-f945e8959c33\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " Jul 28 12:17:34.346076 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.345887 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-online-tls\") pod \"354656be-9109-4915-873f-f945e8959c33\" (UID: \"354656be-9109-4915-873f-f945e8959c33\") " Jul 28 12:17:34.346372 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.346339 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354656be-9109-4915-873f-f945e8959c33-feast-data" (OuterVolumeSpecName: "feast-data") pod "354656be-9109-4915-873f-f945e8959c33" (UID: "354656be-9109-4915-873f-f945e8959c33"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 12:17:34.347901 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.347878 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "354656be-9109-4915-873f-f945e8959c33" (UID: "354656be-9109-4915-873f-f945e8959c33"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:17:34.348007 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.347957 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/354656be-9109-4915-873f-f945e8959c33-kube-api-access-nnp94" (OuterVolumeSpecName: "kube-api-access-nnp94") pod "354656be-9109-4915-873f-f945e8959c33" (UID: "354656be-9109-4915-873f-f945e8959c33"). InnerVolumeSpecName "kube-api-access-nnp94". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 12:17:34.348007 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.347967 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-online-tls" (OuterVolumeSpecName: "online-tls") pod "354656be-9109-4915-873f-f945e8959c33" (UID: "354656be-9109-4915-873f-f945e8959c33"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:17:34.348231 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.348216 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "354656be-9109-4915-873f-f945e8959c33" (UID: "354656be-9109-4915-873f-f945e8959c33"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:17:34.348277 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.348230 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "354656be-9109-4915-873f-f945e8959c33" (UID: "354656be-9109-4915-873f-f945e8959c33"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:17:34.447473 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.447417 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-registry-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:17:34.447473 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.447438 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-offline-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:17:34.447473 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.447448 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnp94\" (UniqueName: \"kubernetes.io/projected/354656be-9109-4915-873f-f945e8959c33-kube-api-access-nnp94\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:17:34.447473 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.447456 2575 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-ui-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:17:34.447473 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.447464 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/354656be-9109-4915-873f-f945e8959c33-online-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:17:34.447473 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:34.447471 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/354656be-9109-4915-873f-f945e8959c33-feast-data\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:17:35.185045 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:35.185005 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" event={"ID":"354656be-9109-4915-873f-f945e8959c33","Type":"ContainerDied","Data":"18ec664fa94a5f0ad6619650b06e1aa23f4cad594e901392e21abc5cef690ec9"} Jul 28 12:17:35.185491 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:35.185063 2575 scope.go:117] "RemoveContainer" containerID="3629434f1040f2c65e8fcc0080ce81c8154dd7ccff75d3e1ce05c2d89367bb84" Jul 28 12:17:35.185491 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:35.185022 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p" Jul 28 12:17:35.199475 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:35.199459 2575 scope.go:117] "RemoveContainer" containerID="2589023fc4abed90f1a857b09be0b51ead3f035c5717daed493e2922cf530ce9" Jul 28 12:17:35.206952 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:35.206924 2575 scope.go:117] "RemoveContainer" containerID="9f88b0c972dac6459c7dde91dba558e76b6d7d12557712d9a86e13e73adbc544" Jul 28 12:17:35.211554 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:35.211527 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p"] Jul 28 12:17:35.216368 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:35.216345 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-66c756fdd-rbs9p"] Jul 28 12:17:35.216522 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:35.216504 2575 scope.go:117] "RemoveContainer" containerID="dd8e257556c0ad5970d63343b57dfa0c130f0a1b7537841fd41c75e57860f506" Jul 28 12:17:35.223564 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:35.223548 2575 scope.go:117] "RemoveContainer" containerID="256945881b6987633a0eb02f26a409feac3d913d7a0aa91b12a97d2a8f662649" Jul 28 12:17:35.232204 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:35.232177 2575 scope.go:117] "RemoveContainer" containerID="43d5d694864654331db120d143951d675623e66400db54474aebec9e2df3a024" Jul 28 12:17:36.209104 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.209075 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="354656be-9109-4915-873f-f945e8959c33" path="/var/lib/kubelet/pods/354656be-9109-4915-873f-f945e8959c33/volumes" Jul 28 12:17:36.569136 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569063 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-9g86r"] Jul 28 12:17:36.569406 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569392 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="feast-init" Jul 28 12:17:36.569464 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569409 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="feast-init" Jul 28 12:17:36.569464 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569423 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="ui" Jul 28 12:17:36.569464 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569435 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="ui" Jul 28 12:17:36.569464 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569441 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="feast-apply" Jul 28 12:17:36.569464 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569448 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="feast-apply" Jul 28 12:17:36.569464 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569455 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="feast-apply" Jul 28 12:17:36.569464 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569460 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="feast-apply" Jul 28 12:17:36.569464 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569467 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="online" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569472 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="online" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569477 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="offline" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569482 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="offline" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569492 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="online" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569497 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="online" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569502 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="ui" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569507 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="ui" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569515 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="feast-init" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569521 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="feast-init" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569531 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="registry" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569536 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="registry" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569543 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="offline" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569550 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="offline" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569602 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="ui" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569610 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="online" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569616 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="registry" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569620 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="354656be-9109-4915-873f-f945e8959c33" containerName="offline" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569626 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="offline" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569634 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="online" Jul 28 12:17:36.569697 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.569640 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7fc7aff-a8cf-4632-a4f7-517dc7b4db42" containerName="ui" Jul 28 12:17:36.574378 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.574357 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-9g86r" Jul 28 12:17:36.577300 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.577275 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 28 12:17:36.578399 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.578372 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-g7tlb\"" Jul 28 12:17:36.578495 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.578378 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 28 12:17:36.581635 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.581613 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-9g86r"] Jul 28 12:17:36.664991 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.664963 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkxsd\" (UniqueName: \"kubernetes.io/projected/b38285ef-4c4c-4dc2-a04d-be54904eafdd-kube-api-access-qkxsd\") pod \"redis-69bbd798b6-9g86r\" (UID: \"b38285ef-4c4c-4dc2-a04d-be54904eafdd\") " pod="test-ns-feast/redis-69bbd798b6-9g86r" Jul 28 12:17:36.766210 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.766186 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkxsd\" (UniqueName: \"kubernetes.io/projected/b38285ef-4c4c-4dc2-a04d-be54904eafdd-kube-api-access-qkxsd\") pod \"redis-69bbd798b6-9g86r\" (UID: \"b38285ef-4c4c-4dc2-a04d-be54904eafdd\") " pod="test-ns-feast/redis-69bbd798b6-9g86r" Jul 28 12:17:36.777569 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.777549 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkxsd\" (UniqueName: \"kubernetes.io/projected/b38285ef-4c4c-4dc2-a04d-be54904eafdd-kube-api-access-qkxsd\") pod \"redis-69bbd798b6-9g86r\" (UID: \"b38285ef-4c4c-4dc2-a04d-be54904eafdd\") " pod="test-ns-feast/redis-69bbd798b6-9g86r" Jul 28 12:17:36.905363 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:36.905271 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-9g86r" Jul 28 12:17:37.030690 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:37.030607 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-9g86r"] Jul 28 12:17:37.032869 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:17:37.032844 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb38285ef_4c4c_4dc2_a04d_be54904eafdd.slice/crio-2255f1fe262c6a90d7b79323b93ff41cc8570f79a377c46d7b1c7d85e7234318 WatchSource:0}: Error finding container 2255f1fe262c6a90d7b79323b93ff41cc8570f79a377c46d7b1c7d85e7234318: Status 404 returned error can't find the container with id 2255f1fe262c6a90d7b79323b93ff41cc8570f79a377c46d7b1c7d85e7234318 Jul 28 12:17:37.034692 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:37.034676 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 12:17:37.195506 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:37.195475 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-9g86r" event={"ID":"b38285ef-4c4c-4dc2-a04d-be54904eafdd","Type":"ContainerStarted","Data":"2255f1fe262c6a90d7b79323b93ff41cc8570f79a377c46d7b1c7d85e7234318"} Jul 28 12:17:39.206282 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:39.206245 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-9g86r" event={"ID":"b38285ef-4c4c-4dc2-a04d-be54904eafdd","Type":"ContainerStarted","Data":"88bf0884f6cf8d1138d9ba33a7b35d8c37edfc15bf7a5fc97f5a231984eb2ac8"} Jul 28 12:17:39.224641 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:39.224592 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-9g86r" podStartSLOduration=1.564163867 podStartE2EDuration="3.224576992s" totalImagesPullingTime="1.660413125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:17:39.223596897 +0000 UTC m=+693.548262177" watchObservedRunningTime="2026-07-28 12:17:39.224576992 +0000 UTC m=+693.549242269" Jul 28 12:17:47.329382 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.329344 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m"] Jul 28 12:17:47.333069 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.333051 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:47.336273 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.336254 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Jul 28 12:17:47.336640 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.336621 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Jul 28 12:17:47.336863 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.336847 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-r2kwp\"" Jul 28 12:17:47.336953 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.336919 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Jul 28 12:17:47.337010 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.336920 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Jul 28 12:17:47.337182 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.337172 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Jul 28 12:17:47.349359 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.349341 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-registry-tls\") pod \"feast-credit-scoring-5b57cf45d8-pxm8m\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:47.349432 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.349367 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5w85\" (UniqueName: \"kubernetes.io/projected/c037c027-23cc-44bd-aa6b-2933fd09f40a-kube-api-access-n5w85\") pod \"feast-credit-scoring-5b57cf45d8-pxm8m\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:47.349432 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.349392 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c037c027-23cc-44bd-aa6b-2933fd09f40a-feast-data\") pod \"feast-credit-scoring-5b57cf45d8-pxm8m\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:47.349512 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.349488 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-online-tls\") pod \"feast-credit-scoring-5b57cf45d8-pxm8m\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:47.349543 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.349514 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-offline-tls\") pod \"feast-credit-scoring-5b57cf45d8-pxm8m\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:47.356605 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.356586 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m"] Jul 28 12:17:47.450352 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.450292 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-online-tls\") pod \"feast-credit-scoring-5b57cf45d8-pxm8m\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:47.450510 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.450358 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-offline-tls\") pod \"feast-credit-scoring-5b57cf45d8-pxm8m\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:47.450510 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.450382 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-registry-tls\") pod \"feast-credit-scoring-5b57cf45d8-pxm8m\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:47.450510 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.450398 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5w85\" (UniqueName: \"kubernetes.io/projected/c037c027-23cc-44bd-aa6b-2933fd09f40a-kube-api-access-n5w85\") pod \"feast-credit-scoring-5b57cf45d8-pxm8m\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:47.450510 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.450421 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c037c027-23cc-44bd-aa6b-2933fd09f40a-feast-data\") pod \"feast-credit-scoring-5b57cf45d8-pxm8m\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:47.450510 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:17:47.450449 2575 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found Jul 28 12:17:47.450789 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:17:47.450525 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-online-tls podName:c037c027-23cc-44bd-aa6b-2933fd09f40a nodeName:}" failed. No retries permitted until 2026-07-28 12:17:47.950507485 +0000 UTC m=+702.275172741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-online-tls") pod "feast-credit-scoring-5b57cf45d8-pxm8m" (UID: "c037c027-23cc-44bd-aa6b-2933fd09f40a") : secret "feast-credit-scoring-online-tls" not found Jul 28 12:17:47.450909 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.450892 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c037c027-23cc-44bd-aa6b-2933fd09f40a-feast-data\") pod \"feast-credit-scoring-5b57cf45d8-pxm8m\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:47.452885 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.452864 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-registry-tls\") pod \"feast-credit-scoring-5b57cf45d8-pxm8m\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:47.452967 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.452898 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-offline-tls\") pod \"feast-credit-scoring-5b57cf45d8-pxm8m\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:47.461448 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.461431 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5w85\" (UniqueName: \"kubernetes.io/projected/c037c027-23cc-44bd-aa6b-2933fd09f40a-kube-api-access-n5w85\") pod \"feast-credit-scoring-5b57cf45d8-pxm8m\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:47.953556 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.953522 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-online-tls\") pod \"feast-credit-scoring-5b57cf45d8-pxm8m\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:47.955892 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:47.955869 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-online-tls\") pod \"feast-credit-scoring-5b57cf45d8-pxm8m\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:48.242710 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:48.242633 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:17:48.376384 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:48.376358 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m"] Jul 28 12:17:48.378823 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:17:48.378801 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc037c027_23cc_44bd_aa6b_2933fd09f40a.slice/crio-95e01006085167c6e88818af6def966679dbc1b5a0f5339969e64d12a6af125c WatchSource:0}: Error finding container 95e01006085167c6e88818af6def966679dbc1b5a0f5339969e64d12a6af125c: Status 404 returned error can't find the container with id 95e01006085167c6e88818af6def966679dbc1b5a0f5339969e64d12a6af125c Jul 28 12:17:49.244823 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:49.244733 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" event={"ID":"c037c027-23cc-44bd-aa6b-2933fd09f40a","Type":"ContainerStarted","Data":"04d030388a82ae56e58960cd0a12ea97c7099bb8d0497618ac9963f8bf582792"} Jul 28 12:17:49.244823 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:49.244790 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" event={"ID":"c037c027-23cc-44bd-aa6b-2933fd09f40a","Type":"ContainerStarted","Data":"95e01006085167c6e88818af6def966679dbc1b5a0f5339969e64d12a6af125c"} Jul 28 12:17:50.256510 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:50.256473 2575 generic.go:356] "Generic (PLEG): container finished" podID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerID="04d030388a82ae56e58960cd0a12ea97c7099bb8d0497618ac9963f8bf582792" exitCode=0 Jul 28 12:17:50.257120 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:50.256525 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" event={"ID":"c037c027-23cc-44bd-aa6b-2933fd09f40a","Type":"ContainerDied","Data":"04d030388a82ae56e58960cd0a12ea97c7099bb8d0497618ac9963f8bf582792"} Jul 28 12:17:51.262965 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:51.262929 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" event={"ID":"c037c027-23cc-44bd-aa6b-2933fd09f40a","Type":"ContainerStarted","Data":"37207f7d9d62fd2841ae38752118dcdcb53400e421ee682efd6fa33142717d5e"} Jul 28 12:17:57.288941 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:57.288910 2575 generic.go:356] "Generic (PLEG): container finished" podID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerID="37207f7d9d62fd2841ae38752118dcdcb53400e421ee682efd6fa33142717d5e" exitCode=0 Jul 28 12:17:57.289340 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:57.288984 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" event={"ID":"c037c027-23cc-44bd-aa6b-2933fd09f40a","Type":"ContainerDied","Data":"37207f7d9d62fd2841ae38752118dcdcb53400e421ee682efd6fa33142717d5e"} Jul 28 12:17:58.298328 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:58.298268 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" event={"ID":"c037c027-23cc-44bd-aa6b-2933fd09f40a","Type":"ContainerStarted","Data":"9fe667c8f9875317ecba6bd7105460aacd816452695346c2216f7f585dad63a6"} Jul 28 12:17:58.298706 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:58.298360 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" event={"ID":"c037c027-23cc-44bd-aa6b-2933fd09f40a","Type":"ContainerStarted","Data":"d9ae357cd2a5a56d3b3d28e836b4e1b9c8dcc05dc663c840188c09bfbf392563"} Jul 28 12:17:58.298706 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:58.298388 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" event={"ID":"c037c027-23cc-44bd-aa6b-2933fd09f40a","Type":"ContainerStarted","Data":"f10255d88cbe69d668b0db597a486d2adf5a36e2830fc9b94964ccd13bd65f4f"} Jul 28 12:17:58.326188 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:17:58.326141 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" podStartSLOduration=11.326125583 podStartE2EDuration="11.326125583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:17:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:17:58.324477468 +0000 UTC m=+712.649142749" watchObservedRunningTime="2026-07-28 12:17:58.326125583 +0000 UTC m=+712.650790861" Jul 28 12:18:00.243157 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:00.243110 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:18:00.243695 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:00.243193 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:18:00.243695 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:00.243221 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:18:00.244869 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:00.244836 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="offline" probeResult="failure" output="dial tcp 10.133.0.31:8816: connect: connection refused" Jul 28 12:18:00.244869 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:00.244856 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.31:6571: connect: connection refused" Jul 28 12:18:00.245076 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:00.244839 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused" Jul 28 12:18:03.244191 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:03.244112 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.31:6571: connect: connection refused" Jul 28 12:18:03.247628 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:03.247600 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:18:03.248708 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:03.248687 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:18:03.249460 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:03.249443 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:18:03.619552 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:03.619473 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:18:04.330135 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:04.330108 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:18:04.333607 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:04.333586 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:18:06.243141 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:06.243105 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.31:6571: connect: connection refused" Jul 28 12:18:08.243741 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:08.243706 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:18:09.244078 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:09.244048 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:18:09.244927 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:18:09.244909 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:19:14.555013 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:14.554973 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m"] Jul 28 12:19:14.555656 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:14.555276 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="registry" containerID="cri-o://f10255d88cbe69d668b0db597a486d2adf5a36e2830fc9b94964ccd13bd65f4f" gracePeriod=30 Jul 28 12:19:14.555656 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:14.555340 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="offline" containerID="cri-o://9fe667c8f9875317ecba6bd7105460aacd816452695346c2216f7f585dad63a6" gracePeriod=30 Jul 28 12:19:14.555656 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:14.555349 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="online" containerID="cri-o://d9ae357cd2a5a56d3b3d28e836b4e1b9c8dcc05dc663c840188c09bfbf392563" gracePeriod=30 Jul 28 12:19:14.605153 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:14.605127 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-9g86r"] Jul 28 12:19:14.605341 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:14.605282 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-9g86r" podUID="b38285ef-4c4c-4dc2-a04d-be54904eafdd" containerName="redis" containerID="cri-o://88bf0884f6cf8d1138d9ba33a7b35d8c37edfc15bf7a5fc97f5a231984eb2ac8" gracePeriod=30 Jul 28 12:19:14.858239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:14.858212 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-9g86r" Jul 28 12:19:14.922953 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:14.922923 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkxsd\" (UniqueName: \"kubernetes.io/projected/b38285ef-4c4c-4dc2-a04d-be54904eafdd-kube-api-access-qkxsd\") pod \"b38285ef-4c4c-4dc2-a04d-be54904eafdd\" (UID: \"b38285ef-4c4c-4dc2-a04d-be54904eafdd\") " Jul 28 12:19:14.925075 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:14.925042 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b38285ef-4c4c-4dc2-a04d-be54904eafdd-kube-api-access-qkxsd" (OuterVolumeSpecName: "kube-api-access-qkxsd") pod "b38285ef-4c4c-4dc2-a04d-be54904eafdd" (UID: "b38285ef-4c4c-4dc2-a04d-be54904eafdd"). InnerVolumeSpecName "kube-api-access-qkxsd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 12:19:15.023534 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:15.023499 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkxsd\" (UniqueName: \"kubernetes.io/projected/b38285ef-4c4c-4dc2-a04d-be54904eafdd-kube-api-access-qkxsd\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:19:15.629266 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:15.629174 2575 generic.go:356] "Generic (PLEG): container finished" podID="b38285ef-4c4c-4dc2-a04d-be54904eafdd" containerID="88bf0884f6cf8d1138d9ba33a7b35d8c37edfc15bf7a5fc97f5a231984eb2ac8" exitCode=0 Jul 28 12:19:15.629266 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:15.629221 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-9g86r" Jul 28 12:19:15.629800 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:15.629264 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-9g86r" event={"ID":"b38285ef-4c4c-4dc2-a04d-be54904eafdd","Type":"ContainerDied","Data":"88bf0884f6cf8d1138d9ba33a7b35d8c37edfc15bf7a5fc97f5a231984eb2ac8"} Jul 28 12:19:15.629800 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:15.629322 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-9g86r" event={"ID":"b38285ef-4c4c-4dc2-a04d-be54904eafdd","Type":"ContainerDied","Data":"2255f1fe262c6a90d7b79323b93ff41cc8570f79a377c46d7b1c7d85e7234318"} Jul 28 12:19:15.629800 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:15.629343 2575 scope.go:117] "RemoveContainer" containerID="88bf0884f6cf8d1138d9ba33a7b35d8c37edfc15bf7a5fc97f5a231984eb2ac8" Jul 28 12:19:15.632456 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:15.632434 2575 generic.go:356] "Generic (PLEG): container finished" podID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerID="d9ae357cd2a5a56d3b3d28e836b4e1b9c8dcc05dc663c840188c09bfbf392563" exitCode=0 Jul 28 12:19:15.632573 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:15.632483 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" event={"ID":"c037c027-23cc-44bd-aa6b-2933fd09f40a","Type":"ContainerDied","Data":"d9ae357cd2a5a56d3b3d28e836b4e1b9c8dcc05dc663c840188c09bfbf392563"} Jul 28 12:19:15.637584 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:15.637567 2575 scope.go:117] "RemoveContainer" containerID="88bf0884f6cf8d1138d9ba33a7b35d8c37edfc15bf7a5fc97f5a231984eb2ac8" Jul 28 12:19:15.637811 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:19:15.637793 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88bf0884f6cf8d1138d9ba33a7b35d8c37edfc15bf7a5fc97f5a231984eb2ac8\": container with ID starting with 88bf0884f6cf8d1138d9ba33a7b35d8c37edfc15bf7a5fc97f5a231984eb2ac8 not found: ID does not exist" containerID="88bf0884f6cf8d1138d9ba33a7b35d8c37edfc15bf7a5fc97f5a231984eb2ac8" Jul 28 12:19:15.637871 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:15.637823 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88bf0884f6cf8d1138d9ba33a7b35d8c37edfc15bf7a5fc97f5a231984eb2ac8"} err="failed to get container status \"88bf0884f6cf8d1138d9ba33a7b35d8c37edfc15bf7a5fc97f5a231984eb2ac8\": rpc error: code = NotFound desc = could not find container \"88bf0884f6cf8d1138d9ba33a7b35d8c37edfc15bf7a5fc97f5a231984eb2ac8\": container with ID starting with 88bf0884f6cf8d1138d9ba33a7b35d8c37edfc15bf7a5fc97f5a231984eb2ac8 not found: ID does not exist" Jul 28 12:19:15.650478 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:15.650458 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-9g86r"] Jul 28 12:19:15.655100 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:15.655079 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-9g86r"] Jul 28 12:19:16.209126 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:16.209094 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b38285ef-4c4c-4dc2-a04d-be54904eafdd" path="/var/lib/kubelet/pods/b38285ef-4c4c-4dc2-a04d-be54904eafdd/volumes" Jul 28 12:19:24.331051 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:24.331011 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused" Jul 28 12:19:34.330538 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:34.330443 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused" Jul 28 12:19:44.330838 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:44.330792 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.31:6567/health\": dial tcp 10.133.0.31:6567: connect: connection refused" Jul 28 12:19:44.331257 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:44.330942 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:19:44.756058 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:44.756021 2575 generic.go:356] "Generic (PLEG): container finished" podID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerID="9fe667c8f9875317ecba6bd7105460aacd816452695346c2216f7f585dad63a6" exitCode=137 Jul 28 12:19:44.756058 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:44.756051 2575 generic.go:356] "Generic (PLEG): container finished" podID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerID="f10255d88cbe69d668b0db597a486d2adf5a36e2830fc9b94964ccd13bd65f4f" exitCode=137 Jul 28 12:19:44.756290 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:44.756098 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" event={"ID":"c037c027-23cc-44bd-aa6b-2933fd09f40a","Type":"ContainerDied","Data":"9fe667c8f9875317ecba6bd7105460aacd816452695346c2216f7f585dad63a6"} Jul 28 12:19:44.756290 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:44.756160 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" event={"ID":"c037c027-23cc-44bd-aa6b-2933fd09f40a","Type":"ContainerDied","Data":"f10255d88cbe69d668b0db597a486d2adf5a36e2830fc9b94964ccd13bd65f4f"} Jul 28 12:19:45.210873 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.210850 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:19:45.271811 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.271779 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c037c027-23cc-44bd-aa6b-2933fd09f40a-feast-data\") pod \"c037c027-23cc-44bd-aa6b-2933fd09f40a\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " Jul 28 12:19:45.272004 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.271835 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5w85\" (UniqueName: \"kubernetes.io/projected/c037c027-23cc-44bd-aa6b-2933fd09f40a-kube-api-access-n5w85\") pod \"c037c027-23cc-44bd-aa6b-2933fd09f40a\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " Jul 28 12:19:45.272004 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.271860 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-registry-tls\") pod \"c037c027-23cc-44bd-aa6b-2933fd09f40a\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " Jul 28 12:19:45.272004 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.271916 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-online-tls\") pod \"c037c027-23cc-44bd-aa6b-2933fd09f40a\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " Jul 28 12:19:45.272004 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.271963 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-offline-tls\") pod \"c037c027-23cc-44bd-aa6b-2933fd09f40a\" (UID: \"c037c027-23cc-44bd-aa6b-2933fd09f40a\") " Jul 28 12:19:45.274129 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.274075 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "c037c027-23cc-44bd-aa6b-2933fd09f40a" (UID: "c037c027-23cc-44bd-aa6b-2933fd09f40a"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:19:45.274250 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.274195 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c037c027-23cc-44bd-aa6b-2933fd09f40a" (UID: "c037c027-23cc-44bd-aa6b-2933fd09f40a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:19:45.274338 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.274245 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c037c027-23cc-44bd-aa6b-2933fd09f40a-kube-api-access-n5w85" (OuterVolumeSpecName: "kube-api-access-n5w85") pod "c037c027-23cc-44bd-aa6b-2933fd09f40a" (UID: "c037c027-23cc-44bd-aa6b-2933fd09f40a"). InnerVolumeSpecName "kube-api-access-n5w85". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 12:19:45.274754 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.274722 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-online-tls" (OuterVolumeSpecName: "online-tls") pod "c037c027-23cc-44bd-aa6b-2933fd09f40a" (UID: "c037c027-23cc-44bd-aa6b-2933fd09f40a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:19:45.281072 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.281052 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c037c027-23cc-44bd-aa6b-2933fd09f40a-feast-data" (OuterVolumeSpecName: "feast-data") pod "c037c027-23cc-44bd-aa6b-2933fd09f40a" (UID: "c037c027-23cc-44bd-aa6b-2933fd09f40a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 12:19:45.372960 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.372865 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5w85\" (UniqueName: \"kubernetes.io/projected/c037c027-23cc-44bd-aa6b-2933fd09f40a-kube-api-access-n5w85\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:19:45.372960 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.372899 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-registry-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:19:45.372960 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.372909 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-online-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:19:45.372960 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.372918 2575 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c037c027-23cc-44bd-aa6b-2933fd09f40a-offline-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:19:45.372960 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.372927 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c037c027-23cc-44bd-aa6b-2933fd09f40a-feast-data\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:19:45.762993 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.762956 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" event={"ID":"c037c027-23cc-44bd-aa6b-2933fd09f40a","Type":"ContainerDied","Data":"95e01006085167c6e88818af6def966679dbc1b5a0f5339969e64d12a6af125c"} Jul 28 12:19:45.762993 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.762983 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m" Jul 28 12:19:45.763376 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.763043 2575 scope.go:117] "RemoveContainer" containerID="9fe667c8f9875317ecba6bd7105460aacd816452695346c2216f7f585dad63a6" Jul 28 12:19:45.772352 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.772336 2575 scope.go:117] "RemoveContainer" containerID="d9ae357cd2a5a56d3b3d28e836b4e1b9c8dcc05dc663c840188c09bfbf392563" Jul 28 12:19:45.779973 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.779954 2575 scope.go:117] "RemoveContainer" containerID="f10255d88cbe69d668b0db597a486d2adf5a36e2830fc9b94964ccd13bd65f4f" Jul 28 12:19:45.787251 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.787236 2575 scope.go:117] "RemoveContainer" containerID="37207f7d9d62fd2841ae38752118dcdcb53400e421ee682efd6fa33142717d5e" Jul 28 12:19:45.792546 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.792525 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m"] Jul 28 12:19:45.794523 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.794499 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-5b57cf45d8-pxm8m"] Jul 28 12:19:45.796383 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:45.796369 2575 scope.go:117] "RemoveContainer" containerID="04d030388a82ae56e58960cd0a12ea97c7099bb8d0497618ac9963f8bf582792" Jul 28 12:19:46.210653 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:19:46.210619 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" path="/var/lib/kubelet/pods/c037c027-23cc-44bd-aa6b-2933fd09f40a/volumes" Jul 28 12:21:06.117883 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:21:06.117848 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/1.log" Jul 28 12:21:06.120175 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:21:06.120153 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/1.log" Jul 28 12:25:40.088972 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.088935 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt"] Jul 28 12:25:40.089421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.089248 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="registry" Jul 28 12:25:40.089421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.089258 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="registry" Jul 28 12:25:40.089421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.089269 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="feast-init" Jul 28 12:25:40.089421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.089274 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="feast-init" Jul 28 12:25:40.089421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.089281 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="feast-apply" Jul 28 12:25:40.089421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.089287 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="feast-apply" Jul 28 12:25:40.089421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.089294 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b38285ef-4c4c-4dc2-a04d-be54904eafdd" containerName="redis" Jul 28 12:25:40.089421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.089300 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b38285ef-4c4c-4dc2-a04d-be54904eafdd" containerName="redis" Jul 28 12:25:40.089421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.089319 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="online" Jul 28 12:25:40.089421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.089323 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="online" Jul 28 12:25:40.089421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.089330 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="offline" Jul 28 12:25:40.089421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.089335 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="offline" Jul 28 12:25:40.089421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.089387 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="online" Jul 28 12:25:40.089421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.089395 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="offline" Jul 28 12:25:40.089421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.089401 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c037c027-23cc-44bd-aa6b-2933fd09f40a" containerName="registry" Jul 28 12:25:40.089421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.089407 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b38285ef-4c4c-4dc2-a04d-be54904eafdd" containerName="redis" Jul 28 12:25:40.092597 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.092577 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:40.095294 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.095260 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Jul 28 12:25:40.095458 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.095321 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Jul 28 12:25:40.095458 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.095358 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Jul 28 12:25:40.095458 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.095360 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Jul 28 12:25:40.095458 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.095359 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Jul 28 12:25:40.095729 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.095713 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Jul 28 12:25:40.095770 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.095739 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-8r779\"" Jul 28 12:25:40.104664 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.104642 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt"] Jul 28 12:25:40.220865 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.220832 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-online-tls\") pod \"feast-credit-scoring-6c8fcd56f-rzbtt\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:40.221047 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.220870 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-registry-tls\") pod \"feast-credit-scoring-6c8fcd56f-rzbtt\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:40.221047 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.220975 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-feast-data\") pod \"feast-credit-scoring-6c8fcd56f-rzbtt\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:40.221047 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.221003 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcd6t\" (UniqueName: \"kubernetes.io/projected/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-kube-api-access-wcd6t\") pod \"feast-credit-scoring-6c8fcd56f-rzbtt\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:40.321840 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.321804 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-feast-data\") pod \"feast-credit-scoring-6c8fcd56f-rzbtt\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:40.321840 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.321840 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcd6t\" (UniqueName: \"kubernetes.io/projected/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-kube-api-access-wcd6t\") pod \"feast-credit-scoring-6c8fcd56f-rzbtt\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:40.322043 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.321978 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-online-tls\") pod \"feast-credit-scoring-6c8fcd56f-rzbtt\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:40.322043 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.322030 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-registry-tls\") pod \"feast-credit-scoring-6c8fcd56f-rzbtt\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:40.322170 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:25:40.322154 2575 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Jul 28 12:25:40.322248 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:25:40.322237 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-registry-tls podName:56fc175c-43f6-4859-9f4a-fc84f41bcb3a nodeName:}" failed. No retries permitted until 2026-07-28 12:25:40.822214965 +0000 UTC m=+1175.146880232 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-registry-tls") pod "feast-credit-scoring-6c8fcd56f-rzbtt" (UID: "56fc175c-43f6-4859-9f4a-fc84f41bcb3a") : secret "feast-credit-scoring-registry-tls" not found Jul 28 12:25:40.322295 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.322283 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-feast-data\") pod \"feast-credit-scoring-6c8fcd56f-rzbtt\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:40.324390 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.324355 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-online-tls\") pod \"feast-credit-scoring-6c8fcd56f-rzbtt\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:40.337043 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.337009 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcd6t\" (UniqueName: \"kubernetes.io/projected/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-kube-api-access-wcd6t\") pod \"feast-credit-scoring-6c8fcd56f-rzbtt\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:40.826207 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.826169 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-registry-tls\") pod \"feast-credit-scoring-6c8fcd56f-rzbtt\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:40.828564 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:40.828533 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-registry-tls\") pod \"feast-credit-scoring-6c8fcd56f-rzbtt\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:41.002262 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:41.002205 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:41.156823 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:41.156794 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt"] Jul 28 12:25:41.158238 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:25:41.158212 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56fc175c_43f6_4859_9f4a_fc84f41bcb3a.slice/crio-71bc9c3d4219a35ffc44288dd29695901b73e1c7d6142eb83ad052cce0042df5 WatchSource:0}: Error finding container 71bc9c3d4219a35ffc44288dd29695901b73e1c7d6142eb83ad052cce0042df5: Status 404 returned error can't find the container with id 71bc9c3d4219a35ffc44288dd29695901b73e1c7d6142eb83ad052cce0042df5 Jul 28 12:25:42.150226 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:42.150188 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" event={"ID":"56fc175c-43f6-4859-9f4a-fc84f41bcb3a","Type":"ContainerStarted","Data":"adc82128332c5b8ca985c4ffc891123e065f0bd0575713c9f26b429b0ac318d2"} Jul 28 12:25:42.150435 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:42.150245 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" event={"ID":"56fc175c-43f6-4859-9f4a-fc84f41bcb3a","Type":"ContainerStarted","Data":"71bc9c3d4219a35ffc44288dd29695901b73e1c7d6142eb83ad052cce0042df5"} Jul 28 12:25:43.155238 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:43.155203 2575 generic.go:356] "Generic (PLEG): container finished" podID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerID="adc82128332c5b8ca985c4ffc891123e065f0bd0575713c9f26b429b0ac318d2" exitCode=0 Jul 28 12:25:43.155673 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:43.155295 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" event={"ID":"56fc175c-43f6-4859-9f4a-fc84f41bcb3a","Type":"ContainerDied","Data":"adc82128332c5b8ca985c4ffc891123e065f0bd0575713c9f26b429b0ac318d2"} Jul 28 12:25:44.161220 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:44.161171 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" event={"ID":"56fc175c-43f6-4859-9f4a-fc84f41bcb3a","Type":"ContainerStarted","Data":"3b10a12c1fcc4cd9d93f616f70ff1b80c1a1a777cb7aab9f9f5aacbc565a5f28"} Jul 28 12:25:51.193095 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:51.193059 2575 generic.go:356] "Generic (PLEG): container finished" podID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerID="3b10a12c1fcc4cd9d93f616f70ff1b80c1a1a777cb7aab9f9f5aacbc565a5f28" exitCode=0 Jul 28 12:25:51.193580 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:51.193138 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" event={"ID":"56fc175c-43f6-4859-9f4a-fc84f41bcb3a","Type":"ContainerDied","Data":"3b10a12c1fcc4cd9d93f616f70ff1b80c1a1a777cb7aab9f9f5aacbc565a5f28"} Jul 28 12:25:52.200561 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:52.200524 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" event={"ID":"56fc175c-43f6-4859-9f4a-fc84f41bcb3a","Type":"ContainerStarted","Data":"9a6260ec06b72b09752d5c0efb2fb7dd70075b3822384dd4af2df4036023b424"} Jul 28 12:25:52.201012 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:52.200584 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" event={"ID":"56fc175c-43f6-4859-9f4a-fc84f41bcb3a","Type":"ContainerStarted","Data":"ef3ba552a6169c43db67f19b80cc62afc367e6f9148428945d51ed6eebbe0c9f"} Jul 28 12:25:52.226229 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:52.226171 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" podStartSLOduration=12.226153725 podStartE2EDuration="12.226153725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:25:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:25:52.223479203 +0000 UTC m=+1186.548144481" watchObservedRunningTime="2026-07-28 12:25:52.226153725 +0000 UTC m=+1186.550819032" Jul 28 12:25:53.002868 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:53.002824 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:53.003062 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:53.002905 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:53.004874 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:53.004834 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.32:6571: connect: connection refused" Jul 28 12:25:53.005001 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:53.004853 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Jul 28 12:25:56.002801 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:56.002747 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.32:6571: connect: connection refused" Jul 28 12:25:56.003266 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:56.002761 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Jul 28 12:25:59.003607 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:59.003565 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.32:6571: connect: connection refused" Jul 28 12:25:59.011634 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:59.011610 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:59.012018 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:59.011994 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:25:59.016363 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:25:59.016338 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:26:01.002654 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:01.002607 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:26:02.003499 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:02.003454 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.32:6571: connect: connection refused" Jul 28 12:26:05.003068 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:05.003039 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:26:05.003788 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:05.003772 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:26:06.144223 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.144195 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/1.log" Jul 28 12:26:06.144679 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.144259 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/1.log" Jul 28 12:26:06.168366 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.168336 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr"] Jul 28 12:26:06.170902 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.170887 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:06.173731 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.173710 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Jul 28 12:26:06.175077 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.175050 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-t2cqw\"" Jul 28 12:26:06.175190 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.175053 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Jul 28 12:26:06.175190 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.175059 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Jul 28 12:26:06.184614 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.184593 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr"] Jul 28 12:26:06.254783 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.254745 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbc4t\" (UniqueName: \"kubernetes.io/projected/914b78ba-4302-4d0e-8f53-9ec1955d69de-kube-api-access-bbc4t\") pod \"feast-driver-ranking-6d777b48c4-j2wcr\" (UID: \"914b78ba-4302-4d0e-8f53-9ec1955d69de\") " pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:06.254783 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.254789 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/914b78ba-4302-4d0e-8f53-9ec1955d69de-registry-tls\") pod \"feast-driver-ranking-6d777b48c4-j2wcr\" (UID: \"914b78ba-4302-4d0e-8f53-9ec1955d69de\") " pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:06.255016 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.254823 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/914b78ba-4302-4d0e-8f53-9ec1955d69de-online-tls\") pod \"feast-driver-ranking-6d777b48c4-j2wcr\" (UID: \"914b78ba-4302-4d0e-8f53-9ec1955d69de\") " pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:06.255016 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.254839 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/914b78ba-4302-4d0e-8f53-9ec1955d69de-feast-data\") pod \"feast-driver-ranking-6d777b48c4-j2wcr\" (UID: \"914b78ba-4302-4d0e-8f53-9ec1955d69de\") " pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:06.355933 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.355897 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/914b78ba-4302-4d0e-8f53-9ec1955d69de-registry-tls\") pod \"feast-driver-ranking-6d777b48c4-j2wcr\" (UID: \"914b78ba-4302-4d0e-8f53-9ec1955d69de\") " pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:06.356139 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.355951 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/914b78ba-4302-4d0e-8f53-9ec1955d69de-online-tls\") pod \"feast-driver-ranking-6d777b48c4-j2wcr\" (UID: \"914b78ba-4302-4d0e-8f53-9ec1955d69de\") " pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:06.356139 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.355975 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/914b78ba-4302-4d0e-8f53-9ec1955d69de-feast-data\") pod \"feast-driver-ranking-6d777b48c4-j2wcr\" (UID: \"914b78ba-4302-4d0e-8f53-9ec1955d69de\") " pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:06.356139 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.356071 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbc4t\" (UniqueName: \"kubernetes.io/projected/914b78ba-4302-4d0e-8f53-9ec1955d69de-kube-api-access-bbc4t\") pod \"feast-driver-ranking-6d777b48c4-j2wcr\" (UID: \"914b78ba-4302-4d0e-8f53-9ec1955d69de\") " pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:06.356494 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.356465 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/914b78ba-4302-4d0e-8f53-9ec1955d69de-feast-data\") pod \"feast-driver-ranking-6d777b48c4-j2wcr\" (UID: \"914b78ba-4302-4d0e-8f53-9ec1955d69de\") " pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:06.358372 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.358346 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/914b78ba-4302-4d0e-8f53-9ec1955d69de-online-tls\") pod \"feast-driver-ranking-6d777b48c4-j2wcr\" (UID: \"914b78ba-4302-4d0e-8f53-9ec1955d69de\") " pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:06.358471 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.358383 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/914b78ba-4302-4d0e-8f53-9ec1955d69de-registry-tls\") pod \"feast-driver-ranking-6d777b48c4-j2wcr\" (UID: \"914b78ba-4302-4d0e-8f53-9ec1955d69de\") " pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:06.368632 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.368605 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbc4t\" (UniqueName: \"kubernetes.io/projected/914b78ba-4302-4d0e-8f53-9ec1955d69de-kube-api-access-bbc4t\") pod \"feast-driver-ranking-6d777b48c4-j2wcr\" (UID: \"914b78ba-4302-4d0e-8f53-9ec1955d69de\") " pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:06.480580 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.480544 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:06.628376 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:06.628346 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr"] Jul 28 12:26:06.630158 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:26:06.630132 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod914b78ba_4302_4d0e_8f53_9ec1955d69de.slice/crio-64bae5284ba62aad62f804782c6540ba7ae2b2aa95188f6cccdf1e535d1c3b73 WatchSource:0}: Error finding container 64bae5284ba62aad62f804782c6540ba7ae2b2aa95188f6cccdf1e535d1c3b73: Status 404 returned error can't find the container with id 64bae5284ba62aad62f804782c6540ba7ae2b2aa95188f6cccdf1e535d1c3b73 Jul 28 12:26:07.262907 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:07.262868 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" event={"ID":"914b78ba-4302-4d0e-8f53-9ec1955d69de","Type":"ContainerStarted","Data":"f65f6e553e070f27727b53a900f9e40061a6c4b5176dbe0951f62f849d188fa3"} Jul 28 12:26:07.263297 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:07.262936 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" event={"ID":"914b78ba-4302-4d0e-8f53-9ec1955d69de","Type":"ContainerStarted","Data":"64bae5284ba62aad62f804782c6540ba7ae2b2aa95188f6cccdf1e535d1c3b73"} Jul 28 12:26:12.288275 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:12.288236 2575 generic.go:356] "Generic (PLEG): container finished" podID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerID="f65f6e553e070f27727b53a900f9e40061a6c4b5176dbe0951f62f849d188fa3" exitCode=0 Jul 28 12:26:12.288728 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:12.288328 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" event={"ID":"914b78ba-4302-4d0e-8f53-9ec1955d69de","Type":"ContainerDied","Data":"f65f6e553e070f27727b53a900f9e40061a6c4b5176dbe0951f62f849d188fa3"} Jul 28 12:26:13.294207 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:13.294169 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" event={"ID":"914b78ba-4302-4d0e-8f53-9ec1955d69de","Type":"ContainerStarted","Data":"62036cd529e0576957e4511d2ba2860af204f0e2164af2f3c0e54a27989daac3"} Jul 28 12:26:20.325678 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:20.325639 2575 generic.go:356] "Generic (PLEG): container finished" podID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerID="62036cd529e0576957e4511d2ba2860af204f0e2164af2f3c0e54a27989daac3" exitCode=0 Jul 28 12:26:20.326066 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:20.325710 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" event={"ID":"914b78ba-4302-4d0e-8f53-9ec1955d69de","Type":"ContainerDied","Data":"62036cd529e0576957e4511d2ba2860af204f0e2164af2f3c0e54a27989daac3"} Jul 28 12:26:21.332817 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:21.332780 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" event={"ID":"914b78ba-4302-4d0e-8f53-9ec1955d69de","Type":"ContainerStarted","Data":"7140591c992c04724a9b7cf0c4bfb66a932f92fe3b20b6b2ed4310e76ca8ca0b"} Jul 28 12:26:21.333203 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:21.332837 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" event={"ID":"914b78ba-4302-4d0e-8f53-9ec1955d69de","Type":"ContainerStarted","Data":"0a1cef337ee269977181789c47e16a6e90fdc8825a92c150632e452114685cd9"} Jul 28 12:26:21.359850 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:21.359778 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" podStartSLOduration=15.359753698 podStartE2EDuration="15.359753698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:26:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:26:21.358645425 +0000 UTC m=+1215.683310728" watchObservedRunningTime="2026-07-28 12:26:21.359753698 +0000 UTC m=+1215.684418965" Jul 28 12:26:21.481695 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:21.481651 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:21.481898 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:21.481743 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:21.483013 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:21.482978 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused" Jul 28 12:26:21.483013 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:21.482997 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused" Jul 28 12:26:24.481617 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:24.481577 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused" Jul 28 12:26:24.482025 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:24.481574 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused" Jul 28 12:26:26.481658 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:26.481617 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:26.482156 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:26.481679 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:27.481025 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:27.480978 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused" Jul 28 12:26:27.489330 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:27.489275 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:27.493903 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:27.493871 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:30.481324 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:30.481253 2575 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="registry" probeResult="failure" output="dial tcp 10.133.0.33:6571: connect: connection refused" Jul 28 12:26:33.482248 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:33.482213 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:26:33.483103 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:26:33.483083 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:27:53.553248 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:53.553206 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7"] Jul 28 12:27:53.553837 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:53.553518 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7" podUID="6c7a5ac7-2229-4d0c-8934-cd13bceca5cd" containerName="manager" containerID="cri-o://84c2c103bfc0738e714102ff5ec6a7b27a0cc7a1e94e60ac0a84884d954fad75" gracePeriod=10 Jul 28 12:27:53.743147 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:53.743115 2575 generic.go:356] "Generic (PLEG): container finished" podID="6c7a5ac7-2229-4d0c-8934-cd13bceca5cd" containerID="84c2c103bfc0738e714102ff5ec6a7b27a0cc7a1e94e60ac0a84884d954fad75" exitCode=0 Jul 28 12:27:53.743327 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:53.743196 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7" event={"ID":"6c7a5ac7-2229-4d0c-8934-cd13bceca5cd","Type":"ContainerDied","Data":"84c2c103bfc0738e714102ff5ec6a7b27a0cc7a1e94e60ac0a84884d954fad75"} Jul 28 12:27:53.828901 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:53.828868 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt"] Jul 28 12:27:53.829173 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:53.829130 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="registry" containerID="cri-o://ef3ba552a6169c43db67f19b80cc62afc367e6f9148428945d51ed6eebbe0c9f" gracePeriod=30 Jul 28 12:27:53.829324 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:53.829190 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="online" containerID="cri-o://9a6260ec06b72b09752d5c0efb2fb7dd70075b3822384dd4af2df4036023b424" gracePeriod=30 Jul 28 12:27:53.831036 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:53.830999 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7" Jul 28 12:27:53.837891 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:53.837857 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr"] Jul 28 12:27:53.838268 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:53.838099 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="registry" containerID="cri-o://0a1cef337ee269977181789c47e16a6e90fdc8825a92c150632e452114685cd9" gracePeriod=30 Jul 28 12:27:53.838268 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:53.838138 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="online" containerID="cri-o://7140591c992c04724a9b7cf0c4bfb66a932f92fe3b20b6b2ed4310e76ca8ca0b" gracePeriod=30 Jul 28 12:27:53.865640 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:53.865600 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6p82\" (UniqueName: \"kubernetes.io/projected/6c7a5ac7-2229-4d0c-8934-cd13bceca5cd-kube-api-access-d6p82\") pod \"6c7a5ac7-2229-4d0c-8934-cd13bceca5cd\" (UID: \"6c7a5ac7-2229-4d0c-8934-cd13bceca5cd\") " Jul 28 12:27:53.867962 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:53.867925 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c7a5ac7-2229-4d0c-8934-cd13bceca5cd-kube-api-access-d6p82" (OuterVolumeSpecName: "kube-api-access-d6p82") pod "6c7a5ac7-2229-4d0c-8934-cd13bceca5cd" (UID: "6c7a5ac7-2229-4d0c-8934-cd13bceca5cd"). InnerVolumeSpecName "kube-api-access-d6p82". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 12:27:53.966535 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:53.966502 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6p82\" (UniqueName: \"kubernetes.io/projected/6c7a5ac7-2229-4d0c-8934-cd13bceca5cd-kube-api-access-d6p82\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:27:54.750583 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:54.750550 2575 generic.go:356] "Generic (PLEG): container finished" podID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerID="9a6260ec06b72b09752d5c0efb2fb7dd70075b3822384dd4af2df4036023b424" exitCode=0 Jul 28 12:27:54.751012 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:54.750621 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" event={"ID":"56fc175c-43f6-4859-9f4a-fc84f41bcb3a","Type":"ContainerDied","Data":"9a6260ec06b72b09752d5c0efb2fb7dd70075b3822384dd4af2df4036023b424"} Jul 28 12:27:54.751778 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:54.751760 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7" Jul 28 12:27:54.751899 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:54.751759 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7" event={"ID":"6c7a5ac7-2229-4d0c-8934-cd13bceca5cd","Type":"ContainerDied","Data":"fceb5d041f085745699147e11c0cac6ce84f5a7f55fb491bd766a90b3badac14"} Jul 28 12:27:54.751899 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:54.751888 2575 scope.go:117] "RemoveContainer" containerID="84c2c103bfc0738e714102ff5ec6a7b27a0cc7a1e94e60ac0a84884d954fad75" Jul 28 12:27:54.754451 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:54.754430 2575 generic.go:356] "Generic (PLEG): container finished" podID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerID="7140591c992c04724a9b7cf0c4bfb66a932f92fe3b20b6b2ed4310e76ca8ca0b" exitCode=0 Jul 28 12:27:54.754537 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:54.754484 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" event={"ID":"914b78ba-4302-4d0e-8f53-9ec1955d69de","Type":"ContainerDied","Data":"7140591c992c04724a9b7cf0c4bfb66a932f92fe3b20b6b2ed4310e76ca8ca0b"} Jul 28 12:27:54.774423 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:54.774401 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7"] Jul 28 12:27:54.779138 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:54.779111 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-f68699c4d-qk2h7"] Jul 28 12:27:56.210170 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:56.210127 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c7a5ac7-2229-4d0c-8934-cd13bceca5cd" path="/var/lib/kubelet/pods/6c7a5ac7-2229-4d0c-8934-cd13bceca5cd/volumes" Jul 28 12:27:57.490178 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:57.490136 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused" Jul 28 12:27:59.013049 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:27:59.013011 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Jul 28 12:28:07.490835 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:07.490785 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused" Jul 28 12:28:09.012239 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:09.012195 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Jul 28 12:28:17.490713 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:17.490669 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="online" probeResult="failure" output="Get \"https://10.133.0.33:6567/health\": dial tcp 10.133.0.33:6567: connect: connection refused" Jul 28 12:28:17.491227 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:17.490795 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:28:19.013009 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:19.012956 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="online" probeResult="failure" output="Get \"https://10.133.0.32:6567/health\": dial tcp 10.133.0.32:6567: connect: connection refused" Jul 28 12:28:19.013525 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:19.013128 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:28:24.510954 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.510927 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:28:24.513480 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.513455 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:28:24.633015 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.632921 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcd6t\" (UniqueName: \"kubernetes.io/projected/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-kube-api-access-wcd6t\") pod \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " Jul 28 12:28:24.633015 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.632957 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/914b78ba-4302-4d0e-8f53-9ec1955d69de-feast-data\") pod \"914b78ba-4302-4d0e-8f53-9ec1955d69de\" (UID: \"914b78ba-4302-4d0e-8f53-9ec1955d69de\") " Jul 28 12:28:24.633015 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.632979 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/914b78ba-4302-4d0e-8f53-9ec1955d69de-online-tls\") pod \"914b78ba-4302-4d0e-8f53-9ec1955d69de\" (UID: \"914b78ba-4302-4d0e-8f53-9ec1955d69de\") " Jul 28 12:28:24.633015 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.633005 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-feast-data\") pod \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " Jul 28 12:28:24.633419 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.633024 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/914b78ba-4302-4d0e-8f53-9ec1955d69de-registry-tls\") pod \"914b78ba-4302-4d0e-8f53-9ec1955d69de\" (UID: \"914b78ba-4302-4d0e-8f53-9ec1955d69de\") " Jul 28 12:28:24.633419 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.633052 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-registry-tls\") pod \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " Jul 28 12:28:24.633419 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.633094 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbc4t\" (UniqueName: \"kubernetes.io/projected/914b78ba-4302-4d0e-8f53-9ec1955d69de-kube-api-access-bbc4t\") pod \"914b78ba-4302-4d0e-8f53-9ec1955d69de\" (UID: \"914b78ba-4302-4d0e-8f53-9ec1955d69de\") " Jul 28 12:28:24.633419 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.633131 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-online-tls\") pod \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\" (UID: \"56fc175c-43f6-4859-9f4a-fc84f41bcb3a\") " Jul 28 12:28:24.633929 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.633883 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914b78ba-4302-4d0e-8f53-9ec1955d69de-feast-data" (OuterVolumeSpecName: "feast-data") pod "914b78ba-4302-4d0e-8f53-9ec1955d69de" (UID: "914b78ba-4302-4d0e-8f53-9ec1955d69de"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 12:28:24.635698 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.635658 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/914b78ba-4302-4d0e-8f53-9ec1955d69de-online-tls" (OuterVolumeSpecName: "online-tls") pod "914b78ba-4302-4d0e-8f53-9ec1955d69de" (UID: "914b78ba-4302-4d0e-8f53-9ec1955d69de"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:28:24.635698 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.635693 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-online-tls" (OuterVolumeSpecName: "online-tls") pod "56fc175c-43f6-4859-9f4a-fc84f41bcb3a" (UID: "56fc175c-43f6-4859-9f4a-fc84f41bcb3a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:28:24.635855 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.635798 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-kube-api-access-wcd6t" (OuterVolumeSpecName: "kube-api-access-wcd6t") pod "56fc175c-43f6-4859-9f4a-fc84f41bcb3a" (UID: "56fc175c-43f6-4859-9f4a-fc84f41bcb3a"). InnerVolumeSpecName "kube-api-access-wcd6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 12:28:24.635855 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.635831 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/914b78ba-4302-4d0e-8f53-9ec1955d69de-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "914b78ba-4302-4d0e-8f53-9ec1955d69de" (UID: "914b78ba-4302-4d0e-8f53-9ec1955d69de"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:28:24.636002 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.635978 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "56fc175c-43f6-4859-9f4a-fc84f41bcb3a" (UID: "56fc175c-43f6-4859-9f4a-fc84f41bcb3a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 12:28:24.636067 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.636048 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/914b78ba-4302-4d0e-8f53-9ec1955d69de-kube-api-access-bbc4t" (OuterVolumeSpecName: "kube-api-access-bbc4t") pod "914b78ba-4302-4d0e-8f53-9ec1955d69de" (UID: "914b78ba-4302-4d0e-8f53-9ec1955d69de"). InnerVolumeSpecName "kube-api-access-bbc4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 12:28:24.643076 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.643054 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-feast-data" (OuterVolumeSpecName: "feast-data") pod "56fc175c-43f6-4859-9f4a-fc84f41bcb3a" (UID: "56fc175c-43f6-4859-9f4a-fc84f41bcb3a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 12:28:24.733747 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.733712 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcd6t\" (UniqueName: \"kubernetes.io/projected/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-kube-api-access-wcd6t\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:28:24.733747 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.733743 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/914b78ba-4302-4d0e-8f53-9ec1955d69de-feast-data\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:28:24.733747 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.733752 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/914b78ba-4302-4d0e-8f53-9ec1955d69de-online-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:28:24.733986 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.733761 2575 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-feast-data\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:28:24.733986 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.733770 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/914b78ba-4302-4d0e-8f53-9ec1955d69de-registry-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:28:24.733986 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.733777 2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-registry-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:28:24.733986 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.733785 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbc4t\" (UniqueName: \"kubernetes.io/projected/914b78ba-4302-4d0e-8f53-9ec1955d69de-kube-api-access-bbc4t\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:28:24.733986 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.733793 2575 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/56fc175c-43f6-4859-9f4a-fc84f41bcb3a-online-tls\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:28:24.882916 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.882879 2575 generic.go:356] "Generic (PLEG): container finished" podID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerID="ef3ba552a6169c43db67f19b80cc62afc367e6f9148428945d51ed6eebbe0c9f" exitCode=137 Jul 28 12:28:24.883158 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.882942 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" Jul 28 12:28:24.883158 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.882965 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" event={"ID":"56fc175c-43f6-4859-9f4a-fc84f41bcb3a","Type":"ContainerDied","Data":"ef3ba552a6169c43db67f19b80cc62afc367e6f9148428945d51ed6eebbe0c9f"} Jul 28 12:28:24.883158 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.883024 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt" event={"ID":"56fc175c-43f6-4859-9f4a-fc84f41bcb3a","Type":"ContainerDied","Data":"71bc9c3d4219a35ffc44288dd29695901b73e1c7d6142eb83ad052cce0042df5"} Jul 28 12:28:24.883158 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.883056 2575 scope.go:117] "RemoveContainer" containerID="9a6260ec06b72b09752d5c0efb2fb7dd70075b3822384dd4af2df4036023b424" Jul 28 12:28:24.885392 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.885367 2575 generic.go:356] "Generic (PLEG): container finished" podID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerID="0a1cef337ee269977181789c47e16a6e90fdc8825a92c150632e452114685cd9" exitCode=137 Jul 28 12:28:24.885542 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.885401 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" event={"ID":"914b78ba-4302-4d0e-8f53-9ec1955d69de","Type":"ContainerDied","Data":"0a1cef337ee269977181789c47e16a6e90fdc8825a92c150632e452114685cd9"} Jul 28 12:28:24.885542 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.885417 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" Jul 28 12:28:24.885542 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.885432 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr" event={"ID":"914b78ba-4302-4d0e-8f53-9ec1955d69de","Type":"ContainerDied","Data":"64bae5284ba62aad62f804782c6540ba7ae2b2aa95188f6cccdf1e535d1c3b73"} Jul 28 12:28:24.893421 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.893402 2575 scope.go:117] "RemoveContainer" containerID="ef3ba552a6169c43db67f19b80cc62afc367e6f9148428945d51ed6eebbe0c9f" Jul 28 12:28:24.901343 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.901325 2575 scope.go:117] "RemoveContainer" containerID="3b10a12c1fcc4cd9d93f616f70ff1b80c1a1a777cb7aab9f9f5aacbc565a5f28" Jul 28 12:28:24.906424 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.906398 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt"] Jul 28 12:28:24.910084 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.910062 2575 scope.go:117] "RemoveContainer" containerID="adc82128332c5b8ca985c4ffc891123e065f0bd0575713c9f26b429b0ac318d2" Jul 28 12:28:24.910769 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.910750 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6c8fcd56f-rzbtt"] Jul 28 12:28:24.917828 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.917810 2575 scope.go:117] "RemoveContainer" containerID="9a6260ec06b72b09752d5c0efb2fb7dd70075b3822384dd4af2df4036023b424" Jul 28 12:28:24.918118 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:28:24.918095 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a6260ec06b72b09752d5c0efb2fb7dd70075b3822384dd4af2df4036023b424\": container with ID starting with 9a6260ec06b72b09752d5c0efb2fb7dd70075b3822384dd4af2df4036023b424 not found: ID does not exist" containerID="9a6260ec06b72b09752d5c0efb2fb7dd70075b3822384dd4af2df4036023b424" Jul 28 12:28:24.918166 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.918129 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a6260ec06b72b09752d5c0efb2fb7dd70075b3822384dd4af2df4036023b424"} err="failed to get container status \"9a6260ec06b72b09752d5c0efb2fb7dd70075b3822384dd4af2df4036023b424\": rpc error: code = NotFound desc = could not find container \"9a6260ec06b72b09752d5c0efb2fb7dd70075b3822384dd4af2df4036023b424\": container with ID starting with 9a6260ec06b72b09752d5c0efb2fb7dd70075b3822384dd4af2df4036023b424 not found: ID does not exist" Jul 28 12:28:24.918166 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.918148 2575 scope.go:117] "RemoveContainer" containerID="ef3ba552a6169c43db67f19b80cc62afc367e6f9148428945d51ed6eebbe0c9f" Jul 28 12:28:24.918420 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:28:24.918403 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef3ba552a6169c43db67f19b80cc62afc367e6f9148428945d51ed6eebbe0c9f\": container with ID starting with ef3ba552a6169c43db67f19b80cc62afc367e6f9148428945d51ed6eebbe0c9f not found: ID does not exist" containerID="ef3ba552a6169c43db67f19b80cc62afc367e6f9148428945d51ed6eebbe0c9f" Jul 28 12:28:24.918471 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.918425 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef3ba552a6169c43db67f19b80cc62afc367e6f9148428945d51ed6eebbe0c9f"} err="failed to get container status \"ef3ba552a6169c43db67f19b80cc62afc367e6f9148428945d51ed6eebbe0c9f\": rpc error: code = NotFound desc = could not find container \"ef3ba552a6169c43db67f19b80cc62afc367e6f9148428945d51ed6eebbe0c9f\": container with ID starting with ef3ba552a6169c43db67f19b80cc62afc367e6f9148428945d51ed6eebbe0c9f not found: ID does not exist" Jul 28 12:28:24.918471 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.918437 2575 scope.go:117] "RemoveContainer" containerID="3b10a12c1fcc4cd9d93f616f70ff1b80c1a1a777cb7aab9f9f5aacbc565a5f28" Jul 28 12:28:24.918674 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:28:24.918655 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b10a12c1fcc4cd9d93f616f70ff1b80c1a1a777cb7aab9f9f5aacbc565a5f28\": container with ID starting with 3b10a12c1fcc4cd9d93f616f70ff1b80c1a1a777cb7aab9f9f5aacbc565a5f28 not found: ID does not exist" containerID="3b10a12c1fcc4cd9d93f616f70ff1b80c1a1a777cb7aab9f9f5aacbc565a5f28" Jul 28 12:28:24.918716 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.918680 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b10a12c1fcc4cd9d93f616f70ff1b80c1a1a777cb7aab9f9f5aacbc565a5f28"} err="failed to get container status \"3b10a12c1fcc4cd9d93f616f70ff1b80c1a1a777cb7aab9f9f5aacbc565a5f28\": rpc error: code = NotFound desc = could not find container \"3b10a12c1fcc4cd9d93f616f70ff1b80c1a1a777cb7aab9f9f5aacbc565a5f28\": container with ID starting with 3b10a12c1fcc4cd9d93f616f70ff1b80c1a1a777cb7aab9f9f5aacbc565a5f28 not found: ID does not exist" Jul 28 12:28:24.918716 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.918696 2575 scope.go:117] "RemoveContainer" containerID="adc82128332c5b8ca985c4ffc891123e065f0bd0575713c9f26b429b0ac318d2" Jul 28 12:28:24.918919 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:28:24.918905 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adc82128332c5b8ca985c4ffc891123e065f0bd0575713c9f26b429b0ac318d2\": container with ID starting with adc82128332c5b8ca985c4ffc891123e065f0bd0575713c9f26b429b0ac318d2 not found: ID does not exist" containerID="adc82128332c5b8ca985c4ffc891123e065f0bd0575713c9f26b429b0ac318d2" Jul 28 12:28:24.918965 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.918923 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adc82128332c5b8ca985c4ffc891123e065f0bd0575713c9f26b429b0ac318d2"} err="failed to get container status \"adc82128332c5b8ca985c4ffc891123e065f0bd0575713c9f26b429b0ac318d2\": rpc error: code = NotFound desc = could not find container \"adc82128332c5b8ca985c4ffc891123e065f0bd0575713c9f26b429b0ac318d2\": container with ID starting with adc82128332c5b8ca985c4ffc891123e065f0bd0575713c9f26b429b0ac318d2 not found: ID does not exist" Jul 28 12:28:24.918965 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.918934 2575 scope.go:117] "RemoveContainer" containerID="7140591c992c04724a9b7cf0c4bfb66a932f92fe3b20b6b2ed4310e76ca8ca0b" Jul 28 12:28:24.923326 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.923279 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr"] Jul 28 12:28:24.927068 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.927052 2575 scope.go:117] "RemoveContainer" containerID="0a1cef337ee269977181789c47e16a6e90fdc8825a92c150632e452114685cd9" Jul 28 12:28:24.927191 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.927174 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6d777b48c4-j2wcr"] Jul 28 12:28:24.934950 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.934934 2575 scope.go:117] "RemoveContainer" containerID="62036cd529e0576957e4511d2ba2860af204f0e2164af2f3c0e54a27989daac3" Jul 28 12:28:24.942313 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.942286 2575 scope.go:117] "RemoveContainer" containerID="f65f6e553e070f27727b53a900f9e40061a6c4b5176dbe0951f62f849d188fa3" Jul 28 12:28:24.949269 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.949247 2575 scope.go:117] "RemoveContainer" containerID="7140591c992c04724a9b7cf0c4bfb66a932f92fe3b20b6b2ed4310e76ca8ca0b" Jul 28 12:28:24.949566 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:28:24.949539 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7140591c992c04724a9b7cf0c4bfb66a932f92fe3b20b6b2ed4310e76ca8ca0b\": container with ID starting with 7140591c992c04724a9b7cf0c4bfb66a932f92fe3b20b6b2ed4310e76ca8ca0b not found: ID does not exist" containerID="7140591c992c04724a9b7cf0c4bfb66a932f92fe3b20b6b2ed4310e76ca8ca0b" Jul 28 12:28:24.949630 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.949575 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7140591c992c04724a9b7cf0c4bfb66a932f92fe3b20b6b2ed4310e76ca8ca0b"} err="failed to get container status \"7140591c992c04724a9b7cf0c4bfb66a932f92fe3b20b6b2ed4310e76ca8ca0b\": rpc error: code = NotFound desc = could not find container \"7140591c992c04724a9b7cf0c4bfb66a932f92fe3b20b6b2ed4310e76ca8ca0b\": container with ID starting with 7140591c992c04724a9b7cf0c4bfb66a932f92fe3b20b6b2ed4310e76ca8ca0b not found: ID does not exist" Jul 28 12:28:24.949630 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.949594 2575 scope.go:117] "RemoveContainer" containerID="0a1cef337ee269977181789c47e16a6e90fdc8825a92c150632e452114685cd9" Jul 28 12:28:24.949880 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:28:24.949862 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a1cef337ee269977181789c47e16a6e90fdc8825a92c150632e452114685cd9\": container with ID starting with 0a1cef337ee269977181789c47e16a6e90fdc8825a92c150632e452114685cd9 not found: ID does not exist" containerID="0a1cef337ee269977181789c47e16a6e90fdc8825a92c150632e452114685cd9" Jul 28 12:28:24.949927 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.949885 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a1cef337ee269977181789c47e16a6e90fdc8825a92c150632e452114685cd9"} err="failed to get container status \"0a1cef337ee269977181789c47e16a6e90fdc8825a92c150632e452114685cd9\": rpc error: code = NotFound desc = could not find container \"0a1cef337ee269977181789c47e16a6e90fdc8825a92c150632e452114685cd9\": container with ID starting with 0a1cef337ee269977181789c47e16a6e90fdc8825a92c150632e452114685cd9 not found: ID does not exist" Jul 28 12:28:24.949927 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.949901 2575 scope.go:117] "RemoveContainer" containerID="62036cd529e0576957e4511d2ba2860af204f0e2164af2f3c0e54a27989daac3" Jul 28 12:28:24.950135 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:28:24.950119 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62036cd529e0576957e4511d2ba2860af204f0e2164af2f3c0e54a27989daac3\": container with ID starting with 62036cd529e0576957e4511d2ba2860af204f0e2164af2f3c0e54a27989daac3 not found: ID does not exist" containerID="62036cd529e0576957e4511d2ba2860af204f0e2164af2f3c0e54a27989daac3" Jul 28 12:28:24.950181 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.950139 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62036cd529e0576957e4511d2ba2860af204f0e2164af2f3c0e54a27989daac3"} err="failed to get container status \"62036cd529e0576957e4511d2ba2860af204f0e2164af2f3c0e54a27989daac3\": rpc error: code = NotFound desc = could not find container \"62036cd529e0576957e4511d2ba2860af204f0e2164af2f3c0e54a27989daac3\": container with ID starting with 62036cd529e0576957e4511d2ba2860af204f0e2164af2f3c0e54a27989daac3 not found: ID does not exist" Jul 28 12:28:24.950181 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.950156 2575 scope.go:117] "RemoveContainer" containerID="f65f6e553e070f27727b53a900f9e40061a6c4b5176dbe0951f62f849d188fa3" Jul 28 12:28:24.950429 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:28:24.950411 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f65f6e553e070f27727b53a900f9e40061a6c4b5176dbe0951f62f849d188fa3\": container with ID starting with f65f6e553e070f27727b53a900f9e40061a6c4b5176dbe0951f62f849d188fa3 not found: ID does not exist" containerID="f65f6e553e070f27727b53a900f9e40061a6c4b5176dbe0951f62f849d188fa3" Jul 28 12:28:24.950515 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:24.950429 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f65f6e553e070f27727b53a900f9e40061a6c4b5176dbe0951f62f849d188fa3"} err="failed to get container status \"f65f6e553e070f27727b53a900f9e40061a6c4b5176dbe0951f62f849d188fa3\": rpc error: code = NotFound desc = could not find container \"f65f6e553e070f27727b53a900f9e40061a6c4b5176dbe0951f62f849d188fa3\": container with ID starting with f65f6e553e070f27727b53a900f9e40061a6c4b5176dbe0951f62f849d188fa3 not found: ID does not exist" Jul 28 12:28:26.209115 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:26.209083 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" path="/var/lib/kubelet/pods/56fc175c-43f6-4859-9f4a-fc84f41bcb3a/volumes" Jul 28 12:28:26.209660 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:28:26.209645 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" path="/var/lib/kubelet/pods/914b78ba-4302-4d0e-8f53-9ec1955d69de/volumes" Jul 28 12:31:06.168454 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:31:06.168425 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/1.log" Jul 28 12:31:06.169410 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:31:06.169386 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/1.log" Jul 28 12:36:06.190806 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:36:06.190773 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/1.log" Jul 28 12:36:06.193129 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:36:06.193107 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/1.log" Jul 28 12:39:00.476047 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476009 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vp627/must-gather-g7n6j"] Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476338 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="online" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476350 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="online" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476360 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="registry" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476365 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="registry" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476374 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="feast-init" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476380 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="feast-init" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476388 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="feast-apply" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476393 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="feast-apply" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476403 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="registry" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476408 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="registry" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476415 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="online" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476419 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="online" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476427 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="feast-init" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476433 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="feast-init" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476439 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c7a5ac7-2229-4d0c-8934-cd13bceca5cd" containerName="manager" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476444 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c7a5ac7-2229-4d0c-8934-cd13bceca5cd" containerName="manager" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476454 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="feast-apply" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476458 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="feast-apply" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476555 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="online" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476566 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="56fc175c-43f6-4859-9f4a-fc84f41bcb3a" containerName="registry" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476573 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="registry" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476579 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c7a5ac7-2229-4d0c-8934-cd13bceca5cd" containerName="manager" Jul 28 12:39:00.476763 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.476585 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="914b78ba-4302-4d0e-8f53-9ec1955d69de" containerName="online" Jul 28 12:39:00.479659 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.479642 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp627/must-gather-g7n6j" Jul 28 12:39:00.482319 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.482291 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vp627\"/\"default-dockercfg-xsx5p\"" Jul 28 12:39:00.483410 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.483390 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vp627\"/\"kube-root-ca.crt\"" Jul 28 12:39:00.483410 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.483403 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vp627\"/\"openshift-service-ca.crt\"" Jul 28 12:39:00.488581 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.488553 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vp627/must-gather-g7n6j"] Jul 28 12:39:00.538556 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.538516 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5c2a457d-51e8-448b-8671-965de5bb57f7-must-gather-output\") pod \"must-gather-g7n6j\" (UID: \"5c2a457d-51e8-448b-8671-965de5bb57f7\") " pod="openshift-must-gather-vp627/must-gather-g7n6j" Jul 28 12:39:00.538748 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.538594 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc5kv\" (UniqueName: \"kubernetes.io/projected/5c2a457d-51e8-448b-8671-965de5bb57f7-kube-api-access-cc5kv\") pod \"must-gather-g7n6j\" (UID: \"5c2a457d-51e8-448b-8671-965de5bb57f7\") " pod="openshift-must-gather-vp627/must-gather-g7n6j" Jul 28 12:39:00.639907 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.639867 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5c2a457d-51e8-448b-8671-965de5bb57f7-must-gather-output\") pod \"must-gather-g7n6j\" (UID: \"5c2a457d-51e8-448b-8671-965de5bb57f7\") " pod="openshift-must-gather-vp627/must-gather-g7n6j" Jul 28 12:39:00.640100 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.639950 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc5kv\" (UniqueName: \"kubernetes.io/projected/5c2a457d-51e8-448b-8671-965de5bb57f7-kube-api-access-cc5kv\") pod \"must-gather-g7n6j\" (UID: \"5c2a457d-51e8-448b-8671-965de5bb57f7\") " pod="openshift-must-gather-vp627/must-gather-g7n6j" Jul 28 12:39:00.640226 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.640206 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5c2a457d-51e8-448b-8671-965de5bb57f7-must-gather-output\") pod \"must-gather-g7n6j\" (UID: \"5c2a457d-51e8-448b-8671-965de5bb57f7\") " pod="openshift-must-gather-vp627/must-gather-g7n6j" Jul 28 12:39:00.648292 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.648267 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc5kv\" (UniqueName: \"kubernetes.io/projected/5c2a457d-51e8-448b-8671-965de5bb57f7-kube-api-access-cc5kv\") pod \"must-gather-g7n6j\" (UID: \"5c2a457d-51e8-448b-8671-965de5bb57f7\") " pod="openshift-must-gather-vp627/must-gather-g7n6j" Jul 28 12:39:00.809121 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.809030 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp627/must-gather-g7n6j" Jul 28 12:39:00.940882 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.940847 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vp627/must-gather-g7n6j"] Jul 28 12:39:00.944016 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:39:00.943986 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c2a457d_51e8_448b_8671_965de5bb57f7.slice/crio-36597fab2fc85fd39afe8faeb83ecdd52239ce2353fbcfb241335d911881659c WatchSource:0}: Error finding container 36597fab2fc85fd39afe8faeb83ecdd52239ce2353fbcfb241335d911881659c: Status 404 returned error can't find the container with id 36597fab2fc85fd39afe8faeb83ecdd52239ce2353fbcfb241335d911881659c Jul 28 12:39:00.945819 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:00.945802 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 12:39:01.298456 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:01.298419 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp627/must-gather-g7n6j" event={"ID":"5c2a457d-51e8-448b-8671-965de5bb57f7","Type":"ContainerStarted","Data":"36597fab2fc85fd39afe8faeb83ecdd52239ce2353fbcfb241335d911881659c"} Jul 28 12:39:08.335043 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:08.335008 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp627/must-gather-g7n6j" event={"ID":"5c2a457d-51e8-448b-8671-965de5bb57f7","Type":"ContainerStarted","Data":"52dba28a5e83b1602e35837413cf7b7cf920c2b319889df9b7d20f26657273ff"} Jul 28 12:39:08.335528 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:08.335056 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp627/must-gather-g7n6j" event={"ID":"5c2a457d-51e8-448b-8671-965de5bb57f7","Type":"ContainerStarted","Data":"0dfad7ba7c012e9910e075f6ce8ef97b4a85ab03ce02a0159239daa0a2ed14de"} Jul 28 12:39:08.351491 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:08.351439 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vp627/must-gather-g7n6j" podStartSLOduration=2.073780969 podStartE2EDuration="8.351423615s" totalImagesPullingTime="6.277642646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 12:39:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:39:08.350289029 +0000 UTC m=+1982.674954308" watchObservedRunningTime="2026-07-28 12:39:08.351423615 +0000 UTC m=+1982.676088894" Jul 28 12:39:17.374567 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:17.374472 2575 generic.go:356] "Generic (PLEG): container finished" podID="5c2a457d-51e8-448b-8671-965de5bb57f7" containerID="0dfad7ba7c012e9910e075f6ce8ef97b4a85ab03ce02a0159239daa0a2ed14de" exitCode=0 Jul 28 12:39:17.374567 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:17.374545 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vp627/must-gather-g7n6j" event={"ID":"5c2a457d-51e8-448b-8671-965de5bb57f7","Type":"ContainerDied","Data":"0dfad7ba7c012e9910e075f6ce8ef97b4a85ab03ce02a0159239daa0a2ed14de"} Jul 28 12:39:17.375107 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:17.374860 2575 scope.go:117] "RemoveContainer" containerID="0dfad7ba7c012e9910e075f6ce8ef97b4a85ab03ce02a0159239daa0a2ed14de" Jul 28 12:39:18.049100 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:18.049063 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vp627_must-gather-g7n6j_5c2a457d-51e8-448b-8671-965de5bb57f7/gather/0.log" Jul 28 12:39:21.147583 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:21.147550 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kkbtd_0f3bc13c-e0e8-44a6-9885-5bcd094112ac/global-pull-secret-syncer/0.log" Jul 28 12:39:21.259491 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:21.259462 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tzfpr_00535017-b0ac-444e-ae72-711bf4003f21/konnectivity-agent/0.log" Jul 28 12:39:21.284843 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:21.284808 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-158.ec2.internal_d1c563166026be471d3364cd537345ce/haproxy/0.log" Jul 28 12:39:23.406662 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:23.406618 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-vp627/must-gather-g7n6j"] Jul 28 12:39:23.407172 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:23.406861 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-vp627/must-gather-g7n6j" podUID="5c2a457d-51e8-448b-8671-965de5bb57f7" containerName="copy" containerID="cri-o://52dba28a5e83b1602e35837413cf7b7cf920c2b319889df9b7d20f26657273ff" gracePeriod=2 Jul 28 12:39:23.409332 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:39:23.409287 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-g7n6j\" is forbidden: User \"system:node:ip-10-0-129-158.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-vp627\": no relationship found between node 'ip-10-0-129-158.ec2.internal' and this object" podUID="5c2a457d-51e8-448b-8671-965de5bb57f7" pod="openshift-must-gather-vp627/must-gather-g7n6j" Jul 28 12:39:23.411276 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:23.410745 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-vp627/must-gather-g7n6j"] Jul 28 12:39:23.640644 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:23.640616 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vp627_must-gather-g7n6j_5c2a457d-51e8-448b-8671-965de5bb57f7/copy/0.log" Jul 28 12:39:23.642339 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:23.642295 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp627/must-gather-g7n6j" Jul 28 12:39:23.644477 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:39:23.644448 2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-g7n6j\" is forbidden: User \"system:node:ip-10-0-129-158.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-vp627\": no relationship found between node 'ip-10-0-129-158.ec2.internal' and this object" podUID="5c2a457d-51e8-448b-8671-965de5bb57f7" pod="openshift-must-gather-vp627/must-gather-g7n6j" Jul 28 12:39:23.741983 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:23.741931 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc5kv\" (UniqueName: \"kubernetes.io/projected/5c2a457d-51e8-448b-8671-965de5bb57f7-kube-api-access-cc5kv\") pod \"5c2a457d-51e8-448b-8671-965de5bb57f7\" (UID: \"5c2a457d-51e8-448b-8671-965de5bb57f7\") " Jul 28 12:39:23.741983 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:23.741984 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5c2a457d-51e8-448b-8671-965de5bb57f7-must-gather-output\") pod \"5c2a457d-51e8-448b-8671-965de5bb57f7\" (UID: \"5c2a457d-51e8-448b-8671-965de5bb57f7\") " Jul 28 12:39:23.742382 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:23.742360 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c2a457d-51e8-448b-8671-965de5bb57f7-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "5c2a457d-51e8-448b-8671-965de5bb57f7" (UID: "5c2a457d-51e8-448b-8671-965de5bb57f7"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 12:39:23.744240 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:23.744219 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c2a457d-51e8-448b-8671-965de5bb57f7-kube-api-access-cc5kv" (OuterVolumeSpecName: "kube-api-access-cc5kv") pod "5c2a457d-51e8-448b-8671-965de5bb57f7" (UID: "5c2a457d-51e8-448b-8671-965de5bb57f7"). InnerVolumeSpecName "kube-api-access-cc5kv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 12:39:23.842909 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:23.842867 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc5kv\" (UniqueName: \"kubernetes.io/projected/5c2a457d-51e8-448b-8671-965de5bb57f7-kube-api-access-cc5kv\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:39:23.842909 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:23.842902 2575 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5c2a457d-51e8-448b-8671-965de5bb57f7-must-gather-output\") on node \"ip-10-0-129-158.ec2.internal\" DevicePath \"\"" Jul 28 12:39:24.208755 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:24.208720 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c2a457d-51e8-448b-8671-965de5bb57f7" path="/var/lib/kubelet/pods/5c2a457d-51e8-448b-8671-965de5bb57f7/volumes" Jul 28 12:39:24.402654 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:24.402624 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-vp627_must-gather-g7n6j_5c2a457d-51e8-448b-8671-965de5bb57f7/copy/0.log" Jul 28 12:39:24.403002 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:24.402979 2575 generic.go:356] "Generic (PLEG): container finished" podID="5c2a457d-51e8-448b-8671-965de5bb57f7" containerID="52dba28a5e83b1602e35837413cf7b7cf920c2b319889df9b7d20f26657273ff" exitCode=143 Jul 28 12:39:24.403069 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:24.403027 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vp627/must-gather-g7n6j" Jul 28 12:39:24.403069 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:24.403054 2575 scope.go:117] "RemoveContainer" containerID="52dba28a5e83b1602e35837413cf7b7cf920c2b319889df9b7d20f26657273ff" Jul 28 12:39:24.410963 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:24.410779 2575 scope.go:117] "RemoveContainer" containerID="0dfad7ba7c012e9910e075f6ce8ef97b4a85ab03ce02a0159239daa0a2ed14de" Jul 28 12:39:24.423042 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:24.423011 2575 scope.go:117] "RemoveContainer" containerID="52dba28a5e83b1602e35837413cf7b7cf920c2b319889df9b7d20f26657273ff" Jul 28 12:39:24.423343 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:39:24.423323 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52dba28a5e83b1602e35837413cf7b7cf920c2b319889df9b7d20f26657273ff\": container with ID starting with 52dba28a5e83b1602e35837413cf7b7cf920c2b319889df9b7d20f26657273ff not found: ID does not exist" containerID="52dba28a5e83b1602e35837413cf7b7cf920c2b319889df9b7d20f26657273ff" Jul 28 12:39:24.423406 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:24.423351 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52dba28a5e83b1602e35837413cf7b7cf920c2b319889df9b7d20f26657273ff"} err="failed to get container status \"52dba28a5e83b1602e35837413cf7b7cf920c2b319889df9b7d20f26657273ff\": rpc error: code = NotFound desc = could not find container \"52dba28a5e83b1602e35837413cf7b7cf920c2b319889df9b7d20f26657273ff\": container with ID starting with 52dba28a5e83b1602e35837413cf7b7cf920c2b319889df9b7d20f26657273ff not found: ID does not exist" Jul 28 12:39:24.423406 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:24.423371 2575 scope.go:117] "RemoveContainer" containerID="0dfad7ba7c012e9910e075f6ce8ef97b4a85ab03ce02a0159239daa0a2ed14de" Jul 28 12:39:24.423593 ip-10-0-129-158 kubenswrapper[2575]: E0728 12:39:24.423574 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dfad7ba7c012e9910e075f6ce8ef97b4a85ab03ce02a0159239daa0a2ed14de\": container with ID starting with 0dfad7ba7c012e9910e075f6ce8ef97b4a85ab03ce02a0159239daa0a2ed14de not found: ID does not exist" containerID="0dfad7ba7c012e9910e075f6ce8ef97b4a85ab03ce02a0159239daa0a2ed14de" Jul 28 12:39:24.423634 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:24.423599 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dfad7ba7c012e9910e075f6ce8ef97b4a85ab03ce02a0159239daa0a2ed14de"} err="failed to get container status \"0dfad7ba7c012e9910e075f6ce8ef97b4a85ab03ce02a0159239daa0a2ed14de\": rpc error: code = NotFound desc = could not find container \"0dfad7ba7c012e9910e075f6ce8ef97b4a85ab03ce02a0159239daa0a2ed14de\": container with ID starting with 0dfad7ba7c012e9910e075f6ce8ef97b4a85ab03ce02a0159239daa0a2ed14de not found: ID does not exist" Jul 28 12:39:25.476621 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:25.476586 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-dn6dz_cff40866-ec36-4069-99af-4409e6f10e55/cluster-monitoring-operator/0.log" Jul 28 12:39:25.506516 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:25.506490 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-hgt58_cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6/kube-state-metrics/0.log" Jul 28 12:39:25.532682 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:25.532653 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-hgt58_cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6/kube-rbac-proxy-main/0.log" Jul 28 12:39:25.561797 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:25.561771 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-hgt58_cb2fac3a-f937-4bb5-98a1-4a7341a2dfd6/kube-rbac-proxy-self/0.log" Jul 28 12:39:25.624725 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:25.624693 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-swwfb_ff2e5d5b-91bb-4e9a-af0b-604bc8a96b1e/monitoring-plugin/0.log" Jul 28 12:39:25.850087 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:25.850003 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rphs9_9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf/node-exporter/0.log" Jul 28 12:39:25.878277 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:25.878252 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rphs9_9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf/kube-rbac-proxy/0.log" Jul 28 12:39:25.904154 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:25.904124 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rphs9_9529fbb6-d1b3-4de4-8ff0-ec3ddef8bcaf/init-textfile/0.log" Jul 28 12:39:26.363921 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:26.363893 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-78mlh_4d8498e4-1b04-492e-8370-2897365d6314/prometheus-operator/0.log" Jul 28 12:39:26.391716 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:26.391684 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-78mlh_4d8498e4-1b04-492e-8370-2897365d6314/kube-rbac-proxy/0.log" Jul 28 12:39:26.459385 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:26.459355 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f5b5f7cfc-zmmnc_facf572b-b208-4a77-b8c7-f6a229807623/telemeter-client/0.log" Jul 28 12:39:26.486193 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:26.486116 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f5b5f7cfc-zmmnc_facf572b-b208-4a77-b8c7-f6a229807623/reload/0.log" Jul 28 12:39:26.514501 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:26.514471 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f5b5f7cfc-zmmnc_facf572b-b208-4a77-b8c7-f6a229807623/kube-rbac-proxy/0.log" Jul 28 12:39:27.702788 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:27.702759 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-86snf_e9c376c1-4429-400e-972c-01b99c92943f/networking-console-plugin/0.log" Jul 28 12:39:28.093493 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.093391 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk"] Jul 28 12:39:28.093784 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.093764 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c2a457d-51e8-448b-8671-965de5bb57f7" containerName="gather" Jul 28 12:39:28.093868 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.093786 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c2a457d-51e8-448b-8671-965de5bb57f7" containerName="gather" Jul 28 12:39:28.093868 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.093813 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c2a457d-51e8-448b-8671-965de5bb57f7" containerName="copy" Jul 28 12:39:28.093868 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.093824 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c2a457d-51e8-448b-8671-965de5bb57f7" containerName="copy" Jul 28 12:39:28.094015 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.093911 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c2a457d-51e8-448b-8671-965de5bb57f7" containerName="copy" Jul 28 12:39:28.094015 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.093929 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c2a457d-51e8-448b-8671-965de5bb57f7" containerName="gather" Jul 28 12:39:28.097578 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.097556 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.100458 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.100435 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8gzv9\"/\"openshift-service-ca.crt\"" Jul 28 12:39:28.100855 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.100839 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8gzv9\"/\"kube-root-ca.crt\"" Jul 28 12:39:28.101694 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.101679 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8gzv9\"/\"default-dockercfg-xd5gj\"" Jul 28 12:39:28.115711 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.115687 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk"] Jul 28 12:39:28.179727 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.179688 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/918d695f-c92c-4a67-a165-cf1e0286f2ae-proc\") pod \"perf-node-gather-daemonset-hrrvk\" (UID: \"918d695f-c92c-4a67-a165-cf1e0286f2ae\") " pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.179924 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.179735 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lp5c\" (UniqueName: \"kubernetes.io/projected/918d695f-c92c-4a67-a165-cf1e0286f2ae-kube-api-access-4lp5c\") pod \"perf-node-gather-daemonset-hrrvk\" (UID: \"918d695f-c92c-4a67-a165-cf1e0286f2ae\") " pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.179924 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.179818 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/918d695f-c92c-4a67-a165-cf1e0286f2ae-podres\") pod \"perf-node-gather-daemonset-hrrvk\" (UID: \"918d695f-c92c-4a67-a165-cf1e0286f2ae\") " pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.179924 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.179857 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/918d695f-c92c-4a67-a165-cf1e0286f2ae-sys\") pod \"perf-node-gather-daemonset-hrrvk\" (UID: \"918d695f-c92c-4a67-a165-cf1e0286f2ae\") " pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.180078 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.179925 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/918d695f-c92c-4a67-a165-cf1e0286f2ae-lib-modules\") pod \"perf-node-gather-daemonset-hrrvk\" (UID: \"918d695f-c92c-4a67-a165-cf1e0286f2ae\") " pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.192739 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.192711 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/1.log" Jul 28 12:39:28.197601 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.197566 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-l4pfh_3dc2810f-7c06-49ad-b0d6-d87a52fd6317/console-operator/2.log" Jul 28 12:39:28.280992 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.280947 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/918d695f-c92c-4a67-a165-cf1e0286f2ae-podres\") pod \"perf-node-gather-daemonset-hrrvk\" (UID: \"918d695f-c92c-4a67-a165-cf1e0286f2ae\") " pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.280992 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.280989 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/918d695f-c92c-4a67-a165-cf1e0286f2ae-sys\") pod \"perf-node-gather-daemonset-hrrvk\" (UID: \"918d695f-c92c-4a67-a165-cf1e0286f2ae\") " pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.281268 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.281086 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/918d695f-c92c-4a67-a165-cf1e0286f2ae-lib-modules\") pod \"perf-node-gather-daemonset-hrrvk\" (UID: \"918d695f-c92c-4a67-a165-cf1e0286f2ae\") " pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.281268 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.281096 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/918d695f-c92c-4a67-a165-cf1e0286f2ae-sys\") pod \"perf-node-gather-daemonset-hrrvk\" (UID: \"918d695f-c92c-4a67-a165-cf1e0286f2ae\") " pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.281268 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.281151 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/918d695f-c92c-4a67-a165-cf1e0286f2ae-podres\") pod \"perf-node-gather-daemonset-hrrvk\" (UID: \"918d695f-c92c-4a67-a165-cf1e0286f2ae\") " pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.281268 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.281225 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/918d695f-c92c-4a67-a165-cf1e0286f2ae-proc\") pod \"perf-node-gather-daemonset-hrrvk\" (UID: \"918d695f-c92c-4a67-a165-cf1e0286f2ae\") " pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.281268 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.281260 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/918d695f-c92c-4a67-a165-cf1e0286f2ae-lib-modules\") pod \"perf-node-gather-daemonset-hrrvk\" (UID: \"918d695f-c92c-4a67-a165-cf1e0286f2ae\") " pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.281517 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.281326 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lp5c\" (UniqueName: \"kubernetes.io/projected/918d695f-c92c-4a67-a165-cf1e0286f2ae-kube-api-access-4lp5c\") pod \"perf-node-gather-daemonset-hrrvk\" (UID: \"918d695f-c92c-4a67-a165-cf1e0286f2ae\") " pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.281517 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.281353 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/918d695f-c92c-4a67-a165-cf1e0286f2ae-proc\") pod \"perf-node-gather-daemonset-hrrvk\" (UID: \"918d695f-c92c-4a67-a165-cf1e0286f2ae\") " pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.290038 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.290006 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lp5c\" (UniqueName: \"kubernetes.io/projected/918d695f-c92c-4a67-a165-cf1e0286f2ae-kube-api-access-4lp5c\") pod \"perf-node-gather-daemonset-hrrvk\" (UID: \"918d695f-c92c-4a67-a165-cf1e0286f2ae\") " pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.407743 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.407650 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:28.539207 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.539171 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk"] Jul 28 12:39:28.543265 ip-10-0-129-158 kubenswrapper[2575]: W0728 12:39:28.543236 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod918d695f_c92c_4a67_a165_cf1e0286f2ae.slice/crio-4966ecad2003b260bb1598a619f8f59cdfb8661c86d6de93b15ca98ed15373aa WatchSource:0}: Error finding container 4966ecad2003b260bb1598a619f8f59cdfb8661c86d6de93b15ca98ed15373aa: Status 404 returned error can't find the container with id 4966ecad2003b260bb1598a619f8f59cdfb8661c86d6de93b15ca98ed15373aa Jul 28 12:39:28.673188 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:28.673162 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-mb9dv_fd8e9c98-3dc1-4d79-934f-82a071887365/download-server/0.log" Jul 28 12:39:29.112279 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:29.112162 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-6zc47_1c2bc53b-e91d-40c4-a177-3724af9c37e9/volume-data-source-validator/0.log" Jul 28 12:39:29.425661 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:29.425626 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" event={"ID":"918d695f-c92c-4a67-a165-cf1e0286f2ae","Type":"ContainerStarted","Data":"353c0467785d53ebe47862745e94c8d94e7a11d0a5b756a0c8b313686c09ebf6"} Jul 28 12:39:29.425835 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:29.425681 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" event={"ID":"918d695f-c92c-4a67-a165-cf1e0286f2ae","Type":"ContainerStarted","Data":"4966ecad2003b260bb1598a619f8f59cdfb8661c86d6de93b15ca98ed15373aa"} Jul 28 12:39:29.425835 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:29.425749 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:29.444942 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:29.444893 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" podStartSLOduration=1.444879406 podStartE2EDuration="1.444879406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 12:39:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 12:39:29.443352468 +0000 UTC m=+2003.768017745" watchObservedRunningTime="2026-07-28 12:39:29.444879406 +0000 UTC m=+2003.769544684" Jul 28 12:39:29.848928 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:29.848822 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6k26v_557e6458-36ad-4c2d-bc32-ba9540692a3f/dns/0.log" Jul 28 12:39:29.869643 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:29.869603 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6k26v_557e6458-36ad-4c2d-bc32-ba9540692a3f/kube-rbac-proxy/0.log" Jul 28 12:39:30.032453 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:30.032419 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n97lp_1c56eeda-c493-4055-87eb-1d2fea4a014d/dns-node-resolver/0.log" Jul 28 12:39:30.430385 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:30.430342 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-f4c495456-mq4qb_fccb616f-b682-4251-9bdb-5b80ba21d6a5/registry/0.log" Jul 28 12:39:30.475619 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:30.475593 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d4d6q_adc764ba-183f-42ff-b0b9-475dcda06640/node-ca/0.log" Jul 28 12:39:31.163431 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:31.163391 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-684fd7d5f9-8xm5z_e2d43bb6-5482-4950-b2a5-b06056f4fa3c/router/0.log" Jul 28 12:39:31.575666 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:31.575637 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x4qcv_1974e756-46ac-42da-886f-8353ebd17e41/serve-healthcheck-canary/0.log" Jul 28 12:39:32.062701 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:32.062652 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lxb9z_c9fc92bb-eabd-4601-8e88-61f3eca0d83a/kube-rbac-proxy/0.log" Jul 28 12:39:32.085937 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:32.085906 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lxb9z_c9fc92bb-eabd-4601-8e88-61f3eca0d83a/exporter/0.log" Jul 28 12:39:32.109614 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:32.109588 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lxb9z_c9fc92bb-eabd-4601-8e88-61f3eca0d83a/extractor/0.log" Jul 28 12:39:35.439985 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:35.439958 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8gzv9/perf-node-gather-daemonset-hrrvk" Jul 28 12:39:37.739885 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:37.739854 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q2dw8_7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425/kube-multus-additional-cni-plugins/0.log" Jul 28 12:39:37.765216 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:37.765190 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q2dw8_7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425/egress-router-binary-copy/0.log" Jul 28 12:39:37.789319 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:37.789275 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q2dw8_7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425/cni-plugins/0.log" Jul 28 12:39:37.814107 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:37.814085 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q2dw8_7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425/bond-cni-plugin/0.log" Jul 28 12:39:37.836281 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:37.836207 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q2dw8_7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425/routeoverride-cni/0.log" Jul 28 12:39:37.857939 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:37.857909 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q2dw8_7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425/whereabouts-cni-bincopy/0.log" Jul 28 12:39:37.878791 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:37.878766 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-q2dw8_7e8d0d2e-73f0-4971-bf1d-cd1fde0b3425/whereabouts-cni/0.log" Jul 28 12:39:38.122877 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:38.122790 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-wnt8s_20658b18-e0e0-4a39-a49a-7b386a1600cb/kube-multus/0.log" Jul 28 12:39:38.141358 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:38.141331 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5f72v_92daaf51-12ba-4eab-87ac-2087d60aac39/network-metrics-daemon/0.log" Jul 28 12:39:38.160944 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:38.160902 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5f72v_92daaf51-12ba-4eab-87ac-2087d60aac39/kube-rbac-proxy/0.log" Jul 28 12:39:39.463363 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:39.463260 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v95xv_ee66aa6f-353c-464a-9a45-a009910fc6e3/ovn-controller/0.log" Jul 28 12:39:39.481402 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:39.481351 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v95xv_ee66aa6f-353c-464a-9a45-a009910fc6e3/ovn-acl-logging/0.log" Jul 28 12:39:39.504157 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:39.504127 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v95xv_ee66aa6f-353c-464a-9a45-a009910fc6e3/kube-rbac-proxy-node/0.log" Jul 28 12:39:39.526701 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:39.526655 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v95xv_ee66aa6f-353c-464a-9a45-a009910fc6e3/kube-rbac-proxy-ovn-metrics/0.log" Jul 28 12:39:39.548511 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:39.548477 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v95xv_ee66aa6f-353c-464a-9a45-a009910fc6e3/northd/0.log" Jul 28 12:39:39.571023 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:39.570998 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v95xv_ee66aa6f-353c-464a-9a45-a009910fc6e3/nbdb/0.log" Jul 28 12:39:39.599064 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:39.599038 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v95xv_ee66aa6f-353c-464a-9a45-a009910fc6e3/sbdb/0.log" Jul 28 12:39:39.649785 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:39.649750 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v95xv_ee66aa6f-353c-464a-9a45-a009910fc6e3/ovnkube-controller/0.log" Jul 28 12:39:40.687567 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:40.687538 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7d88cb55d8-hk9cz_804fcdbc-1bb1-4ca3-9e40-e5ab14d8daad/check-endpoints/0.log" Jul 28 12:39:40.710199 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:40.710172 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-fd7d6_bd0bd704-8198-4017-a6d4-f8222009e54e/network-check-target-container/0.log" Jul 28 12:39:41.632025 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:41.631994 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-276l8_5fe470f0-c27e-418f-af86-7fe9399cd0d4/iptables-alerter/0.log" Jul 28 12:39:42.355412 ip-10-0-129-158 kubenswrapper[2575]: I0728 12:39:42.355318 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-cml7q_bd19a5ef-beec-427f-8a40-5c0f076a8c1d/tuned/0.log"