Jul 28 16:20:58.506260 ip-10-0-140-22 systemd[1]: Starting Kubernetes Kubelet... Jul 28 16:20:58.879373 ip-10-0-140-22 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 16:20:58.879373 ip-10-0-140-22 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 28 16:20:58.879373 ip-10-0-140-22 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 16:20:58.879373 ip-10-0-140-22 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 28 16:20:58.879373 ip-10-0-140-22 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 16:20:58.880954 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.880869 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 28 16:20:58.885060 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885046 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:20:58.885060 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885060 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885064 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885067 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885070 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885073 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885076 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885078 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885081 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885085 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885087 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885090 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885093 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885095 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885098 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885100 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885103 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885105 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885108 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885110 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:20:58.885127 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885114 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885116 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885119 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885122 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885124 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885127 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885129 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885132 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885135 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885137 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885140 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885144 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885148 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885152 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885155 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885159 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885162 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885164 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885168 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:20:58.885579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885170 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885173 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885176 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885178 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885180 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885183 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885185 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885188 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885190 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885193 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885195 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885197 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885199 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885202 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885206 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885209 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885211 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885214 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885217 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885219 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:20:58.886031 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885222 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885225 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885228 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885230 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885233 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885236 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885238 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885241 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885243 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885245 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885248 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885250 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885253 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885256 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885258 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885261 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885263 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885266 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885270 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885273 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:20:58.886486 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885275 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885278 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885281 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885283 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885286 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885288 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885290 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885293 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885296 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885298 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885301 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885303 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885306 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885309 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885311 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885314 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885318 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885321 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885324 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.885326 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:20:58.886942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886058 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886066 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886069 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886073 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886075 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886079 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886082 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886084 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886087 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886089 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886093 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886096 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886098 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886101 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886103 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886106 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886108 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886111 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886114 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886116 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:20:58.887429 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886119 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886121 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886123 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886126 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886128 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886131 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886134 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886136 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886138 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886141 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886145 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886148 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886151 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886154 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886157 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886161 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886180 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886183 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886187 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:20:58.887939 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886190 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886193 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886195 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886198 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886200 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886203 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886205 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886208 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886210 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886213 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886215 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886218 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886220 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886223 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886226 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886228 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886231 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886233 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886236 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:20:58.888443 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886238 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886242 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886246 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886248 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886251 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886253 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886255 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886258 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886260 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886263 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886266 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886268 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886271 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886274 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886276 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886279 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886282 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886284 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886286 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886289 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:20:58.888905 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886291 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886294 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886296 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886299 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886301 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886303 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886306 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886308 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886310 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886312 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886315 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886317 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886320 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886322 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886324 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886326 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886329 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886331 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886334 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886336 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:20:58.889414 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886340 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886689 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886693 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886696 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886699 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886702 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886705 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886708 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886710 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886713 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886715 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886718 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886721 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886723 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886726 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886728 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886730 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886733 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886735 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886737 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:20:58.889882 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886740 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886742 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886745 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886747 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886749 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886752 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886754 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886757 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886759 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886761 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886764 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886766 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886769 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886771 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886774 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886776 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886780 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886782 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886785 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886787 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:20:58.890371 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886790 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886792 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886795 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886810 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886813 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886816 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886819 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886822 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886824 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886827 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886829 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886832 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886836 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886839 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886842 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886844 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886847 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886850 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886853 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:20:58.890831 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886855 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886857 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886860 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886863 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886865 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886868 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886871 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886873 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886875 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886878 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886881 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886883 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886886 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886888 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886891 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886893 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886895 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886898 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886900 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886903 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:20:58.891282 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886905 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886908 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886910 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886913 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886916 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886919 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886921 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886923 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886925 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886929 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886932 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886935 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886938 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886941 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886944 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886947 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886950 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886952 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886954 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:20:58.891772 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886957 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.886959 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887038 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887045 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887051 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887055 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887060 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887063 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887067 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887072 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887075 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887078 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887081 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887085 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887088 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887091 2566 flags.go:64] FLAG: --cgroup-root="" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887093 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887096 2566 flags.go:64] FLAG: --client-ca-file="" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887099 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887102 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887111 2566 flags.go:64] FLAG: --cluster-domain="" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887114 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887116 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887120 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 28 16:20:58.892262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887123 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887126 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887129 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887132 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887135 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887138 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887141 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887147 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887150 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887154 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887158 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887161 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887164 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887167 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887170 2566 flags.go:64] FLAG: --enable-server="true" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887173 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887177 2566 flags.go:64] FLAG: --event-burst="100" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887180 2566 flags.go:64] FLAG: --event-qps="50" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887183 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887186 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887189 2566 flags.go:64] FLAG: --eviction-hard="" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887194 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887196 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887199 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887202 2566 flags.go:64] FLAG: --eviction-soft="" Jul 28 16:20:58.892863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887205 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887207 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887210 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887213 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887216 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887219 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887222 2566 flags.go:64] FLAG: --feature-gates="" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887225 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887228 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887231 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887234 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887237 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887240 2566 flags.go:64] FLAG: --help="false" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887242 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-140-22.ec2.internal" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887247 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887250 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887253 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887256 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887259 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887261 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887264 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887267 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887269 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 28 16:20:58.893501 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887273 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887276 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887279 2566 flags.go:64] FLAG: --kube-reserved="" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887281 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887284 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887288 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887291 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887294 2566 flags.go:64] FLAG: --lock-file="" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887297 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887299 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887302 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887307 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887310 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887313 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887315 2566 flags.go:64] FLAG: --logging-format="text" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887318 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887321 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887324 2566 flags.go:64] FLAG: --manifest-url="" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887327 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887331 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887334 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887338 2566 flags.go:64] FLAG: --max-pods="110" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887341 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887344 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887347 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 28 16:20:58.894107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887350 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887353 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887356 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887359 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887365 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887368 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887371 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887374 2566 flags.go:64] FLAG: --pod-cidr="" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887380 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887385 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887387 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887390 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887394 2566 flags.go:64] FLAG: --port="10250" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887397 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887400 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0cded00329f4084bd" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887403 2566 flags.go:64] FLAG: --qos-reserved="" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887406 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887409 2566 flags.go:64] FLAG: --register-node="true" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887411 2566 flags.go:64] FLAG: --register-with-taints="" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887415 2566 flags.go:64] FLAG: --registry-burst="10" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887417 2566 flags.go:64] FLAG: --registry-qps="5" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887420 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887423 2566 flags.go:64] FLAG: --reserved-memory="" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887426 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887429 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 28 16:20:58.894697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887432 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887435 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887438 2566 flags.go:64] FLAG: --runonce="false" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887441 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887444 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887447 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887451 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887454 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887457 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887460 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887463 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887466 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887468 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887471 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887474 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887477 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887480 2566 flags.go:64] FLAG: --system-cgroups="" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887483 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887488 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887492 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887495 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887499 2566 flags.go:64] FLAG: --tls-min-version="" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887501 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887504 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887507 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 28 16:20:58.895401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887510 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887513 2566 flags.go:64] FLAG: --v="2" Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887517 2566 flags.go:64] FLAG: --version="false" Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887523 2566 flags.go:64] FLAG: --vmodule="" Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887526 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.887529 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887640 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887643 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887646 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887649 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887651 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887654 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887656 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887660 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887663 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887665 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887668 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887670 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887673 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887675 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887677 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:20:58.896007 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887680 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887682 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887685 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887688 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887690 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887694 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887696 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887698 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887700 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887703 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887705 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887707 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887710 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887712 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887714 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887716 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887718 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887721 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887723 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887726 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:20:58.896585 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887728 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887730 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887732 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887735 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887738 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887740 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887743 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887745 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887748 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887750 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887752 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887755 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887757 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887759 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887762 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887764 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887767 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887771 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887773 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:20:58.897160 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887776 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887778 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887780 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887783 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887785 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887787 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887789 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887792 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887794 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887797 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887799 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887801 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887804 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887806 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887809 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887811 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887813 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887817 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887820 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887822 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887824 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:20:58.897628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887827 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887829 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887832 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887836 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887839 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887842 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887845 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887848 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887851 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887855 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887858 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887861 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887863 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887866 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887868 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887871 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887873 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887875 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887877 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:20:58.898131 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887880 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:20:58.898579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887882 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:20:58.898579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887884 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:20:58.898579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887887 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:20:58.898579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.887891 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:20:58.898579 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.888678 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 16:20:58.898579 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.895435 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 28 16:20:58.898579 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.895449 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 28 16:20:58.898579 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.896104 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 28 16:20:58.898579 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.896111 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 28 16:20:58.898579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896172 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:20:58.898579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896176 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:20:58.898579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896179 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:20:58.898579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896182 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:20:58.898579 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896186 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896188 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896191 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896194 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896197 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896207 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896211 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896213 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896218 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896221 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896223 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896226 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896228 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896231 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896233 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896253 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896256 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896258 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896261 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896263 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:20:58.898942 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896266 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896268 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896271 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896273 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896276 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896278 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896281 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896283 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896286 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896288 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896291 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896293 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896295 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896298 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896300 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896303 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896305 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896307 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896310 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:20:58.899454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896312 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896315 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896318 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896320 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896323 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896325 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896327 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896330 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896332 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896335 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896337 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896341 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896344 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896347 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896350 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896353 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896356 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896359 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896362 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:20:58.899925 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896364 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896367 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896370 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896372 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896376 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896380 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896383 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896385 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896388 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896390 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896393 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896395 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896397 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896400 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896402 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896404 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896407 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896410 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896413 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:20:58.900427 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896416 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896419 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896422 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896425 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896427 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896429 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896431 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896434 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896436 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896438 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896441 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896444 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896446 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896448 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896451 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896453 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896456 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:20:58.901040 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896458 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:20:58.901693 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.896463 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 16:20:58.901693 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896596 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:20:58.901693 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896606 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:20:58.901693 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896609 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:20:58.901693 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896611 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:20:58.901693 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896614 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:20:58.901693 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896632 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:20:58.901693 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896635 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:20:58.901693 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896638 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:20:58.901693 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896641 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:20:58.901693 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896644 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:20:58.901693 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896647 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:20:58.901693 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896649 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:20:58.901693 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896653 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:20:58.901693 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896656 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896658 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896661 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896664 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896666 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896669 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896671 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896674 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896676 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896679 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896681 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896683 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896686 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896688 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896690 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896693 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896695 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896697 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896700 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896703 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:20:58.902241 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896705 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896707 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896710 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896712 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896714 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896717 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896719 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896721 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896724 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896726 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896728 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896731 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896734 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896736 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896739 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896741 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896745 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896748 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896751 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896754 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:20:58.902711 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896757 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896759 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896762 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896765 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896767 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896770 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896772 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896775 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896777 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896780 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896783 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896786 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896789 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896792 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896794 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896797 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896799 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896802 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:20:58.903186 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896805 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896809 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896811 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896814 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896816 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896818 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896821 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896823 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896826 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896828 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896831 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896833 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896836 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896838 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896841 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896843 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896846 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896848 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896851 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896853 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:20:58.903606 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896855 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:20:58.904078 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896858 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:20:58.904078 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896860 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:20:58.904078 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896862 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:20:58.904078 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896865 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:20:58.904078 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896867 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:20:58.904078 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896869 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:20:58.904078 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:20:58.896872 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:20:58.904078 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.896876 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 16:20:58.904078 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.897001 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 28 16:20:58.904078 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.898807 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 28 16:20:58.904078 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.900227 2566 server.go:1019] "Starting client certificate rotation" Jul 28 16:20:58.904078 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.900497 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 16:20:58.904078 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.901298 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 16:20:58.904742 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.904725 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 16:20:58.905967 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.905953 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 16:20:58.917604 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.917586 2566 log.go:25] "Validated CRI v1 runtime API" Jul 28 16:20:58.921592 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.921572 2566 log.go:25] "Validated CRI v1 image API" Jul 28 16:20:58.922964 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.922951 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 28 16:20:58.925298 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.925280 2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9ae72796-294f-4206-8276-15c64e500bb6:/dev/nvme0n1p3 b0fc2a50-3e59-4ef6-980c-ff3feacbec3e:/dev/nvme0n1p4] Jul 28 16:20:58.925373 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.925298 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 28 16:20:58.929592 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.929571 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 16:20:58.931691 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.931589 2566 manager.go:216] Machine: {Timestamp:2026-07-28 16:20:58.930357555 +0000 UTC m=+0.312547163 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3172485 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27bd39383cb404bd843d7ed62e91ec SystemUUID:ec27bd39-383c-b404-bd84-3d7ed62e91ec BootID:4be51ca3-904f-4c2d-8247-9e347d729904 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:62:f2:82:df:a9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:62:f2:82:df:a9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:e3:77:1e:6a:09 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 28 16:20:58.931691 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.931692 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 28 16:20:58.931794 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.931766 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 28 16:20:58.934341 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.934317 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 28 16:20:58.934481 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.934344 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-22.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 28 16:20:58.935112 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.935102 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 28 16:20:58.935141 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.935115 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 28 16:20:58.935141 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.935122 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 16:20:58.935875 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.935866 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 16:20:58.936581 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.936572 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 28 16:20:58.938143 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.938132 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 28 16:20:58.938255 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.938246 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 28 16:20:58.938301 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.938293 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 28 16:20:58.938328 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.938308 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 28 16:20:58.938357 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.938335 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 28 16:20:58.938357 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.938348 2566 kubelet.go:398] "Adding apiserver pod source" Jul 28 16:20:58.938407 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.938360 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 28 16:20:58.938789 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.938767 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 16:20:58.938789 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.938791 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 16:20:58.940847 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.940829 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 28 16:20:58.943756 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.943736 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 28 16:20:58.943826 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.943790 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 28 16:20:58.946619 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.946605 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 28 16:20:58.946619 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.946622 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 28 16:20:58.946727 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.946628 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 28 16:20:58.946727 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.946633 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 28 16:20:58.946727 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.946642 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 28 16:20:58.946727 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.946648 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 28 16:20:58.946727 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.946656 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 28 16:20:58.946727 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.946661 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 28 16:20:58.946727 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.946668 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 28 16:20:58.946727 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.946674 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 28 16:20:58.946727 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.946682 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 28 16:20:58.946727 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.946687 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 28 16:20:58.946727 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.946710 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 28 16:20:58.946727 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.946715 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 28 16:20:58.949108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.949093 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-22.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 28 16:20:58.949443 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:58.949423 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-22.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 28 16:20:58.949477 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:58.949436 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 28 16:20:58.949742 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.949732 2566 server.go:1268] "Started kubelet" Jul 28 16:20:58.949991 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.949923 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 28 16:20:58.949991 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.949955 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 28 16:20:58.950311 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.950282 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 28 16:20:58.950626 ip-10-0-140-22 systemd[1]: Started Kubernetes Kubelet. Jul 28 16:20:58.951518 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.951487 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 28 16:20:58.951878 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.951848 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 28 16:20:58.954899 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:58.954874 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 28 16:20:58.957985 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.957956 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 28 16:20:58.958078 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.957989 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 28 16:20:58.958147 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:58.958126 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-22.ec2.internal\" not found" Jul 28 16:20:58.958565 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.958543 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 28 16:20:58.958696 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.958678 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 28 16:20:58.958795 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.958537 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 28 16:20:58.959017 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.958996 2566 factory.go:55] Registering systemd factory Jul 28 16:20:58.959017 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.959016 2566 factory.go:223] Registration of the systemd container factory successfully Jul 28 16:20:58.959152 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.959095 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 28 16:20:58.959152 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.959105 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 28 16:20:58.959286 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.959273 2566 factory.go:153] Registering CRI-O factory Jul 28 16:20:58.959321 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.959290 2566 factory.go:223] Registration of the crio container factory successfully Jul 28 16:20:58.959374 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.959357 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 28 16:20:58.959422 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.959391 2566 factory.go:103] Registering Raw factory Jul 28 16:20:58.959422 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.959405 2566 manager.go:1196] Started watching for new ooms in manager Jul 28 16:20:58.959827 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.959817 2566 manager.go:349] Starting recovery of all containers Jul 28 16:20:58.964399 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:58.964358 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-22.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 28 16:20:58.964899 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:58.964876 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 28 16:20:58.967947 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.967929 2566 manager.go:354] Recovery completed Jul 28 16:20:58.970459 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:58.964147 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-22.ec2.internal.18c6809028f4011a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-22.ec2.internal,UID:ip-10-0-140-22.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-22.ec2.internal,},FirstTimestamp:2026-07-28 16:20:58.94970601 +0000 UTC m=+0.331895619,LastTimestamp:2026-07-28 16:20:58.94970601 +0000 UTC m=+0.331895619,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-22.ec2.internal,}" Jul 28 16:20:58.976278 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.976264 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:20:58.978178 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.978163 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:20:58.978228 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.978191 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:20:58.978228 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.978204 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:20:58.978714 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.978701 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 28 16:20:58.978757 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.978713 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 28 16:20:58.978757 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.978718 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:20:58.978757 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.978729 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 28 16:20:58.979435 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.979420 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:20:58.979506 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.979449 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:20:58.979506 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.979463 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:20:58.980268 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.980233 2566 policy_none.go:49] "None policy: Start" Jul 28 16:20:58.980268 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.980269 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 28 16:20:58.980378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.980283 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 28 16:20:58.980378 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:58.980316 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-22.ec2.internal.18c680902aa6739c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-22.ec2.internal,UID:ip-10-0-140-22.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-22.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-22.ec2.internal,},FirstTimestamp:2026-07-28 16:20:58.978177948 +0000 UTC m=+0.360367558,LastTimestamp:2026-07-28 16:20:58.978177948 +0000 UTC m=+0.360367558,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-22.ec2.internal,}" Jul 28 16:20:58.981273 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.981262 2566 policy_none.go:47] "Start" Jul 28 16:20:58.984529 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.984513 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j6mmz" Jul 28 16:20:58.990030 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:58.989953 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-22.ec2.internal.18c680902aa6bf87 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-22.ec2.internal,UID:ip-10-0-140-22.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-22.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-22.ec2.internal,},FirstTimestamp:2026-07-28 16:20:58.978197383 +0000 UTC m=+0.360386995,LastTimestamp:2026-07-28 16:20:58.978197383 +0000 UTC m=+0.360386995,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-22.ec2.internal,}" Jul 28 16:20:58.991795 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:58.991740 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-22.ec2.internal.18c680902aa6eb51 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-22.ec2.internal,UID:ip-10-0-140-22.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-22.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-22.ec2.internal,},FirstTimestamp:2026-07-28 16:20:58.978208593 +0000 UTC m=+0.360398204,LastTimestamp:2026-07-28 16:20:58.978208593 +0000 UTC m=+0.360398204,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-22.ec2.internal,}" Jul 28 16:20:58.992219 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:58.992193 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j6mmz" Jul 28 16:20:59.018710 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.018553 2566 manager.go:337] "Starting Device Plugin manager" Jul 28 16:20:59.033149 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:59.018729 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 28 16:20:59.033149 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.018744 2566 server.go:87] "Starting device plugin registration server" Jul 28 16:20:59.033149 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.019030 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 28 16:20:59.033149 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.019043 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 28 16:20:59.033149 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.019130 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 28 16:20:59.033149 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.019231 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 28 16:20:59.033149 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.019239 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 28 16:20:59.033149 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:59.019800 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 28 16:20:59.033149 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:59.019851 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-22.ec2.internal\" not found" Jul 28 16:20:59.099964 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.099926 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 28 16:20:59.101307 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.101288 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 28 16:20:59.101449 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.101332 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 28 16:20:59.101449 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.101380 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 28 16:20:59.101449 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:59.101421 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 28 16:20:59.103821 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.103802 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 16:20:59.119581 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.119564 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:20:59.120357 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.120343 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:20:59.120428 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.120371 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:20:59.120428 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.120382 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:20:59.120428 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.120406 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.129240 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.129224 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.129344 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:59.129251 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-22.ec2.internal\": node \"ip-10-0-140-22.ec2.internal\" not found" Jul 28 16:20:59.156166 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:59.156101 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-22.ec2.internal\" not found" Jul 28 16:20:59.201808 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.201767 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-22.ec2.internal"] Jul 28 16:20:59.201945 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.201876 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:20:59.202763 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.202735 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:20:59.202857 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.202768 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:20:59.202857 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.202780 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:20:59.204019 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.204006 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:20:59.204076 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.204055 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.204119 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.204087 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:20:59.204804 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.204786 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:20:59.204804 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.204795 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:20:59.204957 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.204818 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:20:59.204957 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.204834 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:20:59.204957 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.204820 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:20:59.205102 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.204983 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:20:59.206071 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.206057 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.206135 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.206081 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:20:59.206675 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.206662 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:20:59.206760 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.206683 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:20:59.206760 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.206692 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:20:59.231794 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:59.231758 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-22.ec2.internal\" not found" node="ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.236343 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:59.236325 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-22.ec2.internal\" not found" node="ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.256323 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:59.256302 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-22.ec2.internal\" not found" Jul 28 16:20:59.262365 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.262347 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e65469e9ab7d34cf69396aa7763d3b67-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal\" (UID: \"e65469e9ab7d34cf69396aa7763d3b67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.262440 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.262378 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e65469e9ab7d34cf69396aa7763d3b67-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal\" (UID: \"e65469e9ab7d34cf69396aa7763d3b67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.262440 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.262395 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c8a16f2abd34aee43b0a0ab3f7f0846e-config\") pod \"kube-apiserver-proxy-ip-10-0-140-22.ec2.internal\" (UID: \"c8a16f2abd34aee43b0a0ab3f7f0846e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.357396 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:59.357353 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-22.ec2.internal\" not found" Jul 28 16:20:59.362722 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.362703 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e65469e9ab7d34cf69396aa7763d3b67-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal\" (UID: \"e65469e9ab7d34cf69396aa7763d3b67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.362792 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.362729 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e65469e9ab7d34cf69396aa7763d3b67-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal\" (UID: \"e65469e9ab7d34cf69396aa7763d3b67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.362792 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.362746 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c8a16f2abd34aee43b0a0ab3f7f0846e-config\") pod \"kube-apiserver-proxy-ip-10-0-140-22.ec2.internal\" (UID: \"c8a16f2abd34aee43b0a0ab3f7f0846e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.362792 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.362776 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c8a16f2abd34aee43b0a0ab3f7f0846e-config\") pod \"kube-apiserver-proxy-ip-10-0-140-22.ec2.internal\" (UID: \"c8a16f2abd34aee43b0a0ab3f7f0846e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.362889 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.362792 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e65469e9ab7d34cf69396aa7763d3b67-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal\" (UID: \"e65469e9ab7d34cf69396aa7763d3b67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.362889 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.362805 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e65469e9ab7d34cf69396aa7763d3b67-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal\" (UID: \"e65469e9ab7d34cf69396aa7763d3b67\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.458264 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:59.458170 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-22.ec2.internal\" not found" Jul 28 16:20:59.534609 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.534573 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.539614 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.539597 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-22.ec2.internal" Jul 28 16:20:59.558600 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:59.558577 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-22.ec2.internal\" not found" Jul 28 16:20:59.659023 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:59.658991 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-22.ec2.internal\" not found" Jul 28 16:20:59.759389 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:59.759320 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-22.ec2.internal\" not found" Jul 28 16:20:59.859818 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:59.859791 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-22.ec2.internal\" not found" Jul 28 16:20:59.883931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.883903 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 16:20:59.899703 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.899675 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 28 16:20:59.899855 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.899835 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 16:20:59.899912 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.899842 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 16:20:59.959880 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:20:59.959843 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-22.ec2.internal\" not found" Jul 28 16:20:59.996034 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.995996 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 16:15:59 +0000 UTC" deadline="2028-01-17 08:45:15.315192909 +0000 UTC" Jul 28 16:20:59.996034 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:20:59.996025 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12904h24m15.319171061s" Jul 28 16:21:00.028841 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.028819 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 16:21:00.059004 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.058961 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal" Jul 28 16:21:00.072116 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.072083 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 16:21:00.073029 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.073016 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-22.ec2.internal" Jul 28 16:21:00.079296 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.079273 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 16:21:00.118836 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:21:00.118793 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8a16f2abd34aee43b0a0ab3f7f0846e.slice/crio-42ee6e444a134f7d8d2e8655409faebefbceec78e8f4ee2f4c70c9f001d34849 WatchSource:0}: Error finding container 42ee6e444a134f7d8d2e8655409faebefbceec78e8f4ee2f4c70c9f001d34849: Status 404 returned error can't find the container with id 42ee6e444a134f7d8d2e8655409faebefbceec78e8f4ee2f4c70c9f001d34849 Jul 28 16:21:00.119314 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:21:00.119298 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode65469e9ab7d34cf69396aa7763d3b67.slice/crio-0abebfef31aa97892f3e3f8b84963c1da43ea9668fc4669c2a0bf32d05094fb7 WatchSource:0}: Error finding container 0abebfef31aa97892f3e3f8b84963c1da43ea9668fc4669c2a0bf32d05094fb7: Status 404 returned error can't find the container with id 0abebfef31aa97892f3e3f8b84963c1da43ea9668fc4669c2a0bf32d05094fb7 Jul 28 16:21:00.123526 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.123511 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 16:21:00.493433 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.493344 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 16:21:00.939452 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.939363 2566 apiserver.go:52] "Watching apiserver" Jul 28 16:21:00.949789 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.949633 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 28 16:21:00.950125 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.950087 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-nzcls","kube-system/kube-apiserver-proxy-ip-10-0-140-22.ec2.internal","openshift-cluster-node-tuning-operator/tuned-82p66","openshift-dns/node-resolver-8fpkf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal","openshift-multus/multus-lhg5z","openshift-multus/network-metrics-daemon-gb4cw","openshift-network-diagnostics/network-check-target-gbdb4","kube-system/konnectivity-agent-h54gg","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl","openshift-image-registry/node-ca-6jp82","openshift-multus/multus-additional-cni-plugins-knhs8","openshift-network-operator/iptables-alerter-rvhjb","openshift-ovn-kubernetes/ovnkube-node-xx7xl"] Jul 28 16:21:00.951930 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.951900 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:00.953474 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.953450 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.954535 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.954513 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jwpxp\"" Jul 28 16:21:00.954644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.954562 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 28 16:21:00.955456 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.954849 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8fpkf" Jul 28 16:21:00.955456 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.954869 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 28 16:21:00.955456 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.955295 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 28 16:21:00.955914 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.955892 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:00.956042 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:00.955969 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:00.956267 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.956238 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 28 16:21:00.956267 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.956258 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 28 16:21:00.956408 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.956383 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qt455\"" Jul 28 16:21:00.956623 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.956606 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 28 16:21:00.956684 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.956645 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 28 16:21:00.958023 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.957419 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.958023 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.957862 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 28 16:21:00.958023 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.957893 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 28 16:21:00.958023 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.957908 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 28 16:21:00.958296 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.958187 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 28 16:21:00.959083 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.958345 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8jrq5\"" Jul 28 16:21:00.959083 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.958497 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 28 16:21:00.959083 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.958892 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:00.959083 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:00.958954 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:00.960248 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.959761 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 28 16:21:00.960248 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.960022 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 28 16:21:00.960248 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.960231 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 28 16:21:00.960509 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.960494 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:00.960710 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:00.960539 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:00.961078 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.961010 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 28 16:21:00.961596 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.961538 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c89mm\"" Jul 28 16:21:00.964158 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.963785 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h54gg" Jul 28 16:21:00.964916 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.964895 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:00.966456 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.966236 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-b5d9w\"" Jul 28 16:21:00.966634 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.966615 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 28 16:21:00.966771 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.966758 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 28 16:21:00.966891 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.966857 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6jp82" Jul 28 16:21:00.968620 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.967842 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 28 16:21:00.968620 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.968090 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rs8jt\"" Jul 28 16:21:00.968620 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.968197 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:00.968620 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.968282 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 28 16:21:00.971649 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.969731 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 28 16:21:00.971649 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.969948 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 28 16:21:00.971649 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.970156 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 28 16:21:00.971649 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.970332 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zscdj\"" Jul 28 16:21:00.971649 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.970941 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rvhjb" Jul 28 16:21:00.971649 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.971011 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bv6h7\"" Jul 28 16:21:00.971649 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.971305 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 28 16:21:00.971649 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.971471 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 28 16:21:00.972067 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.971748 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tckf\" (UniqueName: \"kubernetes.io/projected/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-kube-api-access-5tckf\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.972067 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.971784 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:00.972067 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.971810 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-var-lib-openvswitch\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.972067 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.971837 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-run-openvswitch\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.972067 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.971860 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-node-log\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.972067 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.971882 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-cni-bin\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.972067 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.971904 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-os-release\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.972067 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.971927 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-multus-socket-dir-parent\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.972067 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.971952 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-systemd-units\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.972067 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.971975 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-run-ovn\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.972067 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972007 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-run-ovn-kubernetes\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.972067 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972066 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.972485 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972100 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/54b3868a-68e2-408f-90a8-808c53e0aefe-env-overrides\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.972485 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972124 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4v2k\" (UniqueName: \"kubernetes.io/projected/54b3868a-68e2-408f-90a8-808c53e0aefe-kube-api-access-j4v2k\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.972485 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972165 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d2caa30-ece8-42cf-9dc4-e45e56dae4a2-hosts-file\") pod \"node-resolver-8fpkf\" (UID: \"4d2caa30-ece8-42cf-9dc4-e45e56dae4a2\") " pod="openshift-dns/node-resolver-8fpkf" Jul 28 16:21:00.972485 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972191 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-socket-dir\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:00.972485 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972214 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-etc-selinux\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:00.972485 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972250 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-kubelet\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.972485 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972275 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d2caa30-ece8-42cf-9dc4-e45e56dae4a2-tmp-dir\") pod \"node-resolver-8fpkf\" (UID: \"4d2caa30-ece8-42cf-9dc4-e45e56dae4a2\") " pod="openshift-dns/node-resolver-8fpkf" Jul 28 16:21:00.972485 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972306 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqgbm\" (UniqueName: \"kubernetes.io/projected/4a8f2962-0386-4066-a462-445cbb6c3f09-kube-api-access-fqgbm\") pod \"network-metrics-daemon-gb4cw\" (UID: \"4a8f2962-0386-4066-a462-445cbb6c3f09\") " pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:00.972485 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972346 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64ljk\" (UniqueName: \"kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk\") pod \"network-check-target-gbdb4\" (UID: \"6d6d1243-b8d6-4402-96f1-5350263231ab\") " pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:00.972485 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972371 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/02ce611d-aef7-4996-bb0f-80279da441b5-kubelet-config\") pod \"global-pull-secret-syncer-nzcls\" (UID: \"02ce611d-aef7-4996-bb0f-80279da441b5\") " pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:00.972485 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972394 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-device-dir\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:00.972485 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972417 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-sys-fs\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:00.972485 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972439 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/54b3868a-68e2-408f-90a8-808c53e0aefe-ovnkube-config\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.972485 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972462 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-system-cni-dir\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.972485 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972486 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-run-k8s-cni-cncf-io\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972514 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-run-netns\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972553 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-var-lib-cni-bin\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972580 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-multus-daemon-config\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972607 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/54b3868a-68e2-408f-90a8-808c53e0aefe-ovn-node-metrics-cert\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972631 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-registration-dir\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972655 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-log-socket\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972686 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/54b3868a-68e2-408f-90a8-808c53e0aefe-ovnkube-script-lib\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972726 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-multus-cni-dir\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972762 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-var-lib-cni-multus\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972788 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-run-multus-certs\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972810 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-etc-kubernetes\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972828 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz8nb\" (UniqueName: \"kubernetes.io/projected/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-kube-api-access-bz8nb\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972853 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-run-systemd\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972869 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-etc-openvswitch\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972884 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-cni-netd\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972899 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/02ce611d-aef7-4996-bb0f-80279da441b5-dbus\") pod \"global-pull-secret-syncer-nzcls\" (UID: \"02ce611d-aef7-4996-bb0f-80279da441b5\") " pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:00.973075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972915 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-cnibin\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.973943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972929 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-hostroot\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.973943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972958 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-multus-conf-dir\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.973943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972975 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn568\" (UniqueName: \"kubernetes.io/projected/4d2caa30-ece8-42cf-9dc4-e45e56dae4a2-kube-api-access-tn568\") pod \"node-resolver-8fpkf\" (UID: \"4d2caa30-ece8-42cf-9dc4-e45e56dae4a2\") " pod="openshift-dns/node-resolver-8fpkf" Jul 28 16:21:00.973943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.972992 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret\") pod \"global-pull-secret-syncer-nzcls\" (UID: \"02ce611d-aef7-4996-bb0f-80279da441b5\") " pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:00.973943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.973010 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-var-lib-kubelet\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.973943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.973026 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-slash\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.973943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.973041 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-run-netns\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:00.973943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.973071 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs\") pod \"network-metrics-daemon-gb4cw\" (UID: \"4a8f2962-0386-4066-a462-445cbb6c3f09\") " pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:00.973943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.973090 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-cni-binary-copy\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:00.973943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.973569 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 28 16:21:00.974459 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.974010 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 28 16:21:00.974459 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.974183 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7bx46\"" Jul 28 16:21:00.974459 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.974378 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 28 16:21:00.976991 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:00.976025 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 16:21:01.002951 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.002926 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-d7gwn" Jul 28 16:21:01.014265 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.014244 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-d7gwn" Jul 28 16:21:01.059977 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.059949 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 28 16:21:01.073396 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.073370 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-cni-netd\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.073642 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.073409 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/faeb9e49-9c13-4f71-9e6c-b14e68624005-cnibin\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.073642 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.073434 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/faeb9e49-9c13-4f71-9e6c-b14e68624005-os-release\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.073642 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.073444 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-cni-netd\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.073642 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.073465 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn568\" (UniqueName: \"kubernetes.io/projected/4d2caa30-ece8-42cf-9dc4-e45e56dae4a2-kube-api-access-tn568\") pod \"node-resolver-8fpkf\" (UID: \"4d2caa30-ece8-42cf-9dc4-e45e56dae4a2\") " pod="openshift-dns/node-resolver-8fpkf" Jul 28 16:21:01.073642 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.073491 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret\") pod \"global-pull-secret-syncer-nzcls\" (UID: \"02ce611d-aef7-4996-bb0f-80279da441b5\") " pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:01.073642 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.073526 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-var-lib-kubelet\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.073642 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.073554 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g7gp\" (UniqueName: \"kubernetes.io/projected/c72d215a-f550-422a-98db-d21bedaca835-kube-api-access-9g7gp\") pod \"iptables-alerter-rvhjb\" (UID: \"c72d215a-f550-422a-98db-d21bedaca835\") " pod="openshift-network-operator/iptables-alerter-rvhjb" Jul 28 16:21:01.073642 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:01.073588 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:21:01.073642 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.073596 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-var-lib-kubelet\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.073642 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.073626 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-cni-binary-copy\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.074146 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:01.073663 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret podName:02ce611d-aef7-4996-bb0f-80279da441b5 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:01.573640968 +0000 UTC m=+2.955830583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret") pod "global-pull-secret-syncer-nzcls" (UID: "02ce611d-aef7-4996-bb0f-80279da441b5") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:21:01.074146 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.073679 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-sysconfig\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.074146 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.073703 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-sysctl-conf\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.074146 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.073931 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f6fa7f51-199b-4804-a481-e8641bad9f5b-agent-certs\") pod \"konnectivity-agent-h54gg\" (UID: \"f6fa7f51-199b-4804-a481-e8641bad9f5b\") " pod="kube-system/konnectivity-agent-h54gg" Jul 28 16:21:01.074146 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.073963 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-var-lib-cni-bin\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.074146 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.073989 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:01.074146 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074012 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-var-lib-openvswitch\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.074146 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074129 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:01.074447 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074165 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-run-openvswitch\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.074447 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074191 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-node-log\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.074447 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074217 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-os-release\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.074447 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074231 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-cni-binary-copy\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.074447 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-multus-socket-dir-parent\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.074447 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074270 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-systemd-units\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.074447 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074424 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-run-ovn\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.074697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074457 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4v2k\" (UniqueName: \"kubernetes.io/projected/54b3868a-68e2-408f-90a8-808c53e0aefe-kube-api-access-j4v2k\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.074697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074484 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-var-lib-cni-bin\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.074697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074483 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64ljk\" (UniqueName: \"kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk\") pod \"network-check-target-gbdb4\" (UID: \"6d6d1243-b8d6-4402-96f1-5350263231ab\") " pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:01.074697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074519 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-modprobe-d\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.074697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074521 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-systemd-units\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.074697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074555 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-kubernetes\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.074697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074560 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-var-lib-openvswitch\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.074697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074580 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-tuned\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.074697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074606 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/faeb9e49-9c13-4f71-9e6c-b14e68624005-tuning-conf-dir\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.074697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074618 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-run-ovn\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.074697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074631 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-socket-dir\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:01.074697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074654 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-run-openvswitch\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.074697 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074697 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-node-log\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.075201 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074752 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-os-release\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.075201 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074880 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-multus-socket-dir-parent\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.075201 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.074965 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-socket-dir\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:01.075201 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075098 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d2caa30-ece8-42cf-9dc4-e45e56dae4a2-tmp-dir\") pod \"node-resolver-8fpkf\" (UID: \"4d2caa30-ece8-42cf-9dc4-e45e56dae4a2\") " pod="openshift-dns/node-resolver-8fpkf" Jul 28 16:21:01.075201 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075137 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqgbm\" (UniqueName: \"kubernetes.io/projected/4a8f2962-0386-4066-a462-445cbb6c3f09-kube-api-access-fqgbm\") pod \"network-metrics-daemon-gb4cw\" (UID: \"4a8f2962-0386-4066-a462-445cbb6c3f09\") " pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:01.075201 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075189 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c72d215a-f550-422a-98db-d21bedaca835-host-slash\") pod \"iptables-alerter-rvhjb\" (UID: \"c72d215a-f550-422a-98db-d21bedaca835\") " pod="openshift-network-operator/iptables-alerter-rvhjb" Jul 28 16:21:01.075437 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075240 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4258901a-a0ea-48aa-921d-a3217ab8a119-tmp\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.075437 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075265 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f6fa7f51-199b-4804-a481-e8641bad9f5b-konnectivity-ca\") pod \"konnectivity-agent-h54gg\" (UID: \"f6fa7f51-199b-4804-a481-e8641bad9f5b\") " pod="kube-system/konnectivity-agent-h54gg" Jul 28 16:21:01.075437 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075292 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/faeb9e49-9c13-4f71-9e6c-b14e68624005-system-cni-dir\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.075437 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075340 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/54b3868a-68e2-408f-90a8-808c53e0aefe-ovn-node-metrics-cert\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.075437 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075366 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-log-socket\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.075437 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075400 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-run\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.075437 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075412 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d2caa30-ece8-42cf-9dc4-e45e56dae4a2-tmp-dir\") pod \"node-resolver-8fpkf\" (UID: \"4d2caa30-ece8-42cf-9dc4-e45e56dae4a2\") " pod="openshift-dns/node-resolver-8fpkf" Jul 28 16:21:01.075693 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075425 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5mdr\" (UniqueName: \"kubernetes.io/projected/4258901a-a0ea-48aa-921d-a3217ab8a119-kube-api-access-j5mdr\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.075693 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075466 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c72d215a-f550-422a-98db-d21bedaca835-iptables-alerter-script\") pod \"iptables-alerter-rvhjb\" (UID: \"c72d215a-f550-422a-98db-d21bedaca835\") " pod="openshift-network-operator/iptables-alerter-rvhjb" Jul 28 16:21:01.075693 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075506 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/02ce611d-aef7-4996-bb0f-80279da441b5-dbus\") pod \"global-pull-secret-syncer-nzcls\" (UID: \"02ce611d-aef7-4996-bb0f-80279da441b5\") " pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:01.075693 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075531 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-cnibin\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.075693 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075554 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-hostroot\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.075693 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075581 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-multus-conf-dir\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.075693 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075621 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-systemd\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.075693 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075647 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-slash\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.075693 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075671 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-run-netns\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.076177 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075699 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs\") pod \"network-metrics-daemon-gb4cw\" (UID: \"4a8f2962-0386-4066-a462-445cbb6c3f09\") " pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:01.076177 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075724 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tckf\" (UniqueName: \"kubernetes.io/projected/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-kube-api-access-5tckf\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.076177 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075748 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gf7n\" (UniqueName: \"kubernetes.io/projected/18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b-kube-api-access-9gf7n\") pod \"node-ca-6jp82\" (UID: \"18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b\") " pod="openshift-image-registry/node-ca-6jp82" Jul 28 16:21:01.076177 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075795 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-run-netns\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.076177 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075799 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 28 16:21:01.076177 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075858 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-cni-bin\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.076177 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075887 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-run-ovn-kubernetes\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.076177 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075916 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.076177 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075945 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/54b3868a-68e2-408f-90a8-808c53e0aefe-env-overrides\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.076177 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.075991 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d2caa30-ece8-42cf-9dc4-e45e56dae4a2-hosts-file\") pod \"node-resolver-8fpkf\" (UID: \"4d2caa30-ece8-42cf-9dc4-e45e56dae4a2\") " pod="openshift-dns/node-resolver-8fpkf" Jul 28 16:21:01.076177 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076065 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/02ce611d-aef7-4996-bb0f-80279da441b5-kubelet-config\") pod \"global-pull-secret-syncer-nzcls\" (UID: \"02ce611d-aef7-4996-bb0f-80279da441b5\") " pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:01.076177 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076102 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-var-lib-cni-multus\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.076177 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076122 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/02ce611d-aef7-4996-bb0f-80279da441b5-dbus\") pod \"global-pull-secret-syncer-nzcls\" (UID: \"02ce611d-aef7-4996-bb0f-80279da441b5\") " pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:01.076177 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076152 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-sysctl-d\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076195 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-cnibin\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076213 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-run-netns\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076253 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-multus-conf-dir\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076238 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-hostroot\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076289 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-slash\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076322 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d2caa30-ece8-42cf-9dc4-e45e56dae4a2-hosts-file\") pod \"node-resolver-8fpkf\" (UID: \"4d2caa30-ece8-42cf-9dc4-e45e56dae4a2\") " pod="openshift-dns/node-resolver-8fpkf" Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076369 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-cni-bin\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:01.076399 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076405 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-run-ovn-kubernetes\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076325 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-run-netns\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:01.076496 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs podName:4a8f2962-0386-4066-a462-445cbb6c3f09 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:01.576449606 +0000 UTC m=+2.958639227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs") pod "network-metrics-daemon-gb4cw" (UID: "4a8f2962-0386-4066-a462-445cbb6c3f09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076550 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076593 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-etc-selinux\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076622 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-kubelet\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076651 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-multus-daemon-config\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076664 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/02ce611d-aef7-4996-bb0f-80279da441b5-kubelet-config\") pod \"global-pull-secret-syncer-nzcls\" (UID: \"02ce611d-aef7-4996-bb0f-80279da441b5\") " pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:01.076903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076681 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-run-multus-certs\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076726 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-sys\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076743 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-lib-modules\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076760 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-var-lib-kubelet\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076781 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-device-dir\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076799 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-sys-fs\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076817 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/54b3868a-68e2-408f-90a8-808c53e0aefe-ovnkube-config\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076833 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-system-cni-dir\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076870 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-run-k8s-cni-cncf-io\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076888 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-etc-kubernetes\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076909 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-host\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076924 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b-serviceca\") pod \"node-ca-6jp82\" (UID: \"18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b\") " pod="openshift-image-registry/node-ca-6jp82" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076958 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-registration-dir\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076978 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/54b3868a-68e2-408f-90a8-808c53e0aefe-ovnkube-script-lib\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.076994 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-multus-cni-dir\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077008 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b-host\") pod \"node-ca-6jp82\" (UID: \"18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b\") " pod="openshift-image-registry/node-ca-6jp82" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077026 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/faeb9e49-9c13-4f71-9e6c-b14e68624005-cni-binary-copy\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.077898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077076 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/faeb9e49-9c13-4f71-9e6c-b14e68624005-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077102 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/faeb9e49-9c13-4f71-9e6c-b14e68624005-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077129 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsmvt\" (UniqueName: \"kubernetes.io/projected/faeb9e49-9c13-4f71-9e6c-b14e68624005-kube-api-access-qsmvt\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077152 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz8nb\" (UniqueName: \"kubernetes.io/projected/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-kube-api-access-bz8nb\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077170 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-run-systemd\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077205 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-multus-daemon-config\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077232 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-etc-openvswitch\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077262 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-run-multus-certs\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077299 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-etc-openvswitch\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077312 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-run-k8s-cni-cncf-io\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077322 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-host-kubelet\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077357 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-etc-kubernetes\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077364 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-run-systemd\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077388 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-registration-dir\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077427 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-system-cni-dir\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077825 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/54b3868a-68e2-408f-90a8-808c53e0aefe-env-overrides\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077852 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-host-var-lib-cni-multus\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.078644 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077886 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-device-dir\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:01.079339 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077901 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/54b3868a-68e2-408f-90a8-808c53e0aefe-ovnkube-script-lib\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.079339 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077907 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/54b3868a-68e2-408f-90a8-808c53e0aefe-log-socket\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.079339 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077859 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-multus-cni-dir\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.079339 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077932 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-sys-fs\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:01.079339 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.077950 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-etc-selinux\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:01.079339 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.078541 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/54b3868a-68e2-408f-90a8-808c53e0aefe-ovnkube-config\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.080392 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.080367 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/54b3868a-68e2-408f-90a8-808c53e0aefe-ovn-node-metrics-cert\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.081897 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:01.081873 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:21:01.082000 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:01.081904 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:21:01.082000 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:01.081919 2566 projected.go:196] Error preparing data for projected volume kube-api-access-64ljk for pod openshift-network-diagnostics/network-check-target-gbdb4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:21:01.082141 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:01.082005 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk podName:6d6d1243-b8d6-4402-96f1-5350263231ab nodeName:}" failed. No retries permitted until 2026-07-28 16:21:01.581952327 +0000 UTC m=+2.964141938 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-64ljk" (UniqueName: "kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk") pod "network-check-target-gbdb4" (UID: "6d6d1243-b8d6-4402-96f1-5350263231ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:21:01.084932 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.084901 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4v2k\" (UniqueName: \"kubernetes.io/projected/54b3868a-68e2-408f-90a8-808c53e0aefe-kube-api-access-j4v2k\") pod \"ovnkube-node-xx7xl\" (UID: \"54b3868a-68e2-408f-90a8-808c53e0aefe\") " pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.085395 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.085373 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn568\" (UniqueName: \"kubernetes.io/projected/4d2caa30-ece8-42cf-9dc4-e45e56dae4a2-kube-api-access-tn568\") pod \"node-resolver-8fpkf\" (UID: \"4d2caa30-ece8-42cf-9dc4-e45e56dae4a2\") " pod="openshift-dns/node-resolver-8fpkf" Jul 28 16:21:01.085470 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.085443 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tckf\" (UniqueName: \"kubernetes.io/projected/312acc96-6bbf-4d5e-98fd-f5142efd2bdd-kube-api-access-5tckf\") pod \"multus-lhg5z\" (UID: \"312acc96-6bbf-4d5e-98fd-f5142efd2bdd\") " pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.087338 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.087297 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz8nb\" (UniqueName: \"kubernetes.io/projected/ecee27e5-8dc3-4b95-a111-4cfa81feea7a-kube-api-access-bz8nb\") pod \"aws-ebs-csi-driver-node-fmxvl\" (UID: \"ecee27e5-8dc3-4b95-a111-4cfa81feea7a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:01.087438 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.087400 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqgbm\" (UniqueName: \"kubernetes.io/projected/4a8f2962-0386-4066-a462-445cbb6c3f09-kube-api-access-fqgbm\") pod \"network-metrics-daemon-gb4cw\" (UID: \"4a8f2962-0386-4066-a462-445cbb6c3f09\") " pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:01.106578 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.106513 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal" event={"ID":"e65469e9ab7d34cf69396aa7763d3b67","Type":"ContainerStarted","Data":"0abebfef31aa97892f3e3f8b84963c1da43ea9668fc4669c2a0bf32d05094fb7"} Jul 28 16:21:01.108222 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.108176 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-22.ec2.internal" event={"ID":"c8a16f2abd34aee43b0a0ab3f7f0846e","Type":"ContainerStarted","Data":"42ee6e444a134f7d8d2e8655409faebefbceec78e8f4ee2f4c70c9f001d34849"} Jul 28 16:21:01.178250 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178207 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gf7n\" (UniqueName: \"kubernetes.io/projected/18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b-kube-api-access-9gf7n\") pod \"node-ca-6jp82\" (UID: \"18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b\") " pod="openshift-image-registry/node-ca-6jp82" Jul 28 16:21:01.178250 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178252 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-sysctl-d\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.178447 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178276 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-sys\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.178447 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178299 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-lib-modules\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.178447 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178322 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-var-lib-kubelet\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.178447 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178383 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-var-lib-kubelet\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.178447 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178390 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-sys\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.178658 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178468 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-sysctl-d\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.178658 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178506 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-lib-modules\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.178658 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178510 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-host\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.178658 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178544 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b-serviceca\") pod \"node-ca-6jp82\" (UID: \"18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b\") " pod="openshift-image-registry/node-ca-6jp82" Jul 28 16:21:01.178658 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178575 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-host\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.178658 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178585 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b-host\") pod \"node-ca-6jp82\" (UID: \"18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b\") " pod="openshift-image-registry/node-ca-6jp82" Jul 28 16:21:01.178658 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178620 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b-host\") pod \"node-ca-6jp82\" (UID: \"18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b\") " pod="openshift-image-registry/node-ca-6jp82" Jul 28 16:21:01.178658 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178637 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/faeb9e49-9c13-4f71-9e6c-b14e68624005-cni-binary-copy\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.178658 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178659 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/faeb9e49-9c13-4f71-9e6c-b14e68624005-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178677 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/faeb9e49-9c13-4f71-9e6c-b14e68624005-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178694 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsmvt\" (UniqueName: \"kubernetes.io/projected/faeb9e49-9c13-4f71-9e6c-b14e68624005-kube-api-access-qsmvt\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178715 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/faeb9e49-9c13-4f71-9e6c-b14e68624005-cnibin\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178738 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/faeb9e49-9c13-4f71-9e6c-b14e68624005-os-release\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178778 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g7gp\" (UniqueName: \"kubernetes.io/projected/c72d215a-f550-422a-98db-d21bedaca835-kube-api-access-9g7gp\") pod \"iptables-alerter-rvhjb\" (UID: \"c72d215a-f550-422a-98db-d21bedaca835\") " pod="openshift-network-operator/iptables-alerter-rvhjb" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178802 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-sysconfig\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178828 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-sysctl-conf\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178850 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f6fa7f51-199b-4804-a481-e8641bad9f5b-agent-certs\") pod \"konnectivity-agent-h54gg\" (UID: \"f6fa7f51-199b-4804-a481-e8641bad9f5b\") " pod="kube-system/konnectivity-agent-h54gg" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178910 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-modprobe-d\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178933 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-kubernetes\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178958 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-tuned\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.178985 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/faeb9e49-9c13-4f71-9e6c-b14e68624005-tuning-conf-dir\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179022 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/faeb9e49-9c13-4f71-9e6c-b14e68624005-cnibin\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179031 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c72d215a-f550-422a-98db-d21bedaca835-host-slash\") pod \"iptables-alerter-rvhjb\" (UID: \"c72d215a-f550-422a-98db-d21bedaca835\") " pod="openshift-network-operator/iptables-alerter-rvhjb" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179056 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4258901a-a0ea-48aa-921d-a3217ab8a119-tmp\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179078 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f6fa7f51-199b-4804-a481-e8641bad9f5b-konnectivity-ca\") pod \"konnectivity-agent-h54gg\" (UID: \"f6fa7f51-199b-4804-a481-e8641bad9f5b\") " pod="kube-system/konnectivity-agent-h54gg" Jul 28 16:21:01.179108 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179102 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/faeb9e49-9c13-4f71-9e6c-b14e68624005-system-cni-dir\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179108 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b-serviceca\") pod \"node-ca-6jp82\" (UID: \"18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b\") " pod="openshift-image-registry/node-ca-6jp82" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179117 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-modprobe-d\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179128 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-run\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179179 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-run\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179191 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/faeb9e49-9c13-4f71-9e6c-b14e68624005-os-release\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179219 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c72d215a-f550-422a-98db-d21bedaca835-host-slash\") pod \"iptables-alerter-rvhjb\" (UID: \"c72d215a-f550-422a-98db-d21bedaca835\") " pod="openshift-network-operator/iptables-alerter-rvhjb" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179220 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5mdr\" (UniqueName: \"kubernetes.io/projected/4258901a-a0ea-48aa-921d-a3217ab8a119-kube-api-access-j5mdr\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179252 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c72d215a-f550-422a-98db-d21bedaca835-iptables-alerter-script\") pod \"iptables-alerter-rvhjb\" (UID: \"c72d215a-f550-422a-98db-d21bedaca835\") " pod="openshift-network-operator/iptables-alerter-rvhjb" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179280 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-systemd\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179278 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/faeb9e49-9c13-4f71-9e6c-b14e68624005-cni-binary-copy\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179417 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-sysctl-conf\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179471 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/faeb9e49-9c13-4f71-9e6c-b14e68624005-system-cni-dir\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179519 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-sysconfig\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179561 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/faeb9e49-9c13-4f71-9e6c-b14e68624005-tuning-conf-dir\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179630 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-systemd\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179865 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/faeb9e49-9c13-4f71-9e6c-b14e68624005-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.179931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179880 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-kubernetes\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.181015 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179891 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/faeb9e49-9c13-4f71-9e6c-b14e68624005-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.181015 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.179916 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f6fa7f51-199b-4804-a481-e8641bad9f5b-konnectivity-ca\") pod \"konnectivity-agent-h54gg\" (UID: \"f6fa7f51-199b-4804-a481-e8641bad9f5b\") " pod="kube-system/konnectivity-agent-h54gg" Jul 28 16:21:01.181015 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.180107 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c72d215a-f550-422a-98db-d21bedaca835-iptables-alerter-script\") pod \"iptables-alerter-rvhjb\" (UID: \"c72d215a-f550-422a-98db-d21bedaca835\") " pod="openshift-network-operator/iptables-alerter-rvhjb" Jul 28 16:21:01.181909 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.181884 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4258901a-a0ea-48aa-921d-a3217ab8a119-tmp\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.182243 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.182205 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4258901a-a0ea-48aa-921d-a3217ab8a119-etc-tuned\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.182887 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.182693 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f6fa7f51-199b-4804-a481-e8641bad9f5b-agent-certs\") pod \"konnectivity-agent-h54gg\" (UID: \"f6fa7f51-199b-4804-a481-e8641bad9f5b\") " pod="kube-system/konnectivity-agent-h54gg" Jul 28 16:21:01.186978 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.186946 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsmvt\" (UniqueName: \"kubernetes.io/projected/faeb9e49-9c13-4f71-9e6c-b14e68624005-kube-api-access-qsmvt\") pod \"multus-additional-cni-plugins-knhs8\" (UID: \"faeb9e49-9c13-4f71-9e6c-b14e68624005\") " pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.187358 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.187337 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gf7n\" (UniqueName: \"kubernetes.io/projected/18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b-kube-api-access-9gf7n\") pod \"node-ca-6jp82\" (UID: \"18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b\") " pod="openshift-image-registry/node-ca-6jp82" Jul 28 16:21:01.187487 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.187459 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g7gp\" (UniqueName: \"kubernetes.io/projected/c72d215a-f550-422a-98db-d21bedaca835-kube-api-access-9g7gp\") pod \"iptables-alerter-rvhjb\" (UID: \"c72d215a-f550-422a-98db-d21bedaca835\") " pod="openshift-network-operator/iptables-alerter-rvhjb" Jul 28 16:21:01.188418 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.188387 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5mdr\" (UniqueName: \"kubernetes.io/projected/4258901a-a0ea-48aa-921d-a3217ab8a119-kube-api-access-j5mdr\") pod \"tuned-82p66\" (UID: \"4258901a-a0ea-48aa-921d-a3217ab8a119\") " pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.266729 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.266647 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" Jul 28 16:21:01.286242 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.286214 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:01.292149 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.292125 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8fpkf" Jul 28 16:21:01.298063 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.298032 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lhg5z" Jul 28 16:21:01.305908 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.305885 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h54gg" Jul 28 16:21:01.321236 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.321209 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-82p66" Jul 28 16:21:01.330196 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.330160 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6jp82" Jul 28 16:21:01.339333 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.339305 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-knhs8" Jul 28 16:21:01.346310 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.346289 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rvhjb" Jul 28 16:21:01.482422 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.482388 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 16:21:01.582493 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.582387 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret\") pod \"global-pull-secret-syncer-nzcls\" (UID: \"02ce611d-aef7-4996-bb0f-80279da441b5\") " pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:01.582493 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.582458 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64ljk\" (UniqueName: \"kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk\") pod \"network-check-target-gbdb4\" (UID: \"6d6d1243-b8d6-4402-96f1-5350263231ab\") " pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:01.582493 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:01.582505 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs\") pod \"network-metrics-daemon-gb4cw\" (UID: \"4a8f2962-0386-4066-a462-445cbb6c3f09\") " pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:01.582787 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:01.582585 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:21:01.582787 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:01.582623 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:21:01.582787 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:01.582639 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:21:01.582787 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:01.582651 2566 projected.go:196] Error preparing data for projected volume kube-api-access-64ljk for pod openshift-network-diagnostics/network-check-target-gbdb4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:21:01.582787 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:01.582664 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret podName:02ce611d-aef7-4996-bb0f-80279da441b5 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:02.58264414 +0000 UTC m=+3.964833757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret") pod "global-pull-secret-syncer-nzcls" (UID: "02ce611d-aef7-4996-bb0f-80279da441b5") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:21:01.582787 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:01.582590 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:21:01.582787 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:01.582707 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk podName:6d6d1243-b8d6-4402-96f1-5350263231ab nodeName:}" failed. No retries permitted until 2026-07-28 16:21:02.582691781 +0000 UTC m=+3.964881399 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-64ljk" (UniqueName: "kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk") pod "network-check-target-gbdb4" (UID: "6d6d1243-b8d6-4402-96f1-5350263231ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:21:01.582787 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:01.582726 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs podName:4a8f2962-0386-4066-a462-445cbb6c3f09 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:02.582716812 +0000 UTC m=+3.964906422 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs") pod "network-metrics-daemon-gb4cw" (UID: "4a8f2962-0386-4066-a462-445cbb6c3f09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:21:02.017092 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:02.016992 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 16:16:01 +0000 UTC" deadline="2028-02-11 18:38:07.9293614 +0000 UTC" Jul 28 16:21:02.017092 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:02.017041 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13514h17m5.912324072s" Jul 28 16:21:02.102158 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:02.102117 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:02.102158 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:02.102157 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:02.102357 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:02.102244 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:02.102401 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:02.102369 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:02.119057 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:21:02.118910 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18769dd9_4ec7_45b1_994f_6d0d3bd4aa8b.slice/crio-4993b2cea624756aa22f5b824ad5dabd4d22876d6fb86aa95c3d22d163d2ff46 WatchSource:0}: Error finding container 4993b2cea624756aa22f5b824ad5dabd4d22876d6fb86aa95c3d22d163d2ff46: Status 404 returned error can't find the container with id 4993b2cea624756aa22f5b824ad5dabd4d22876d6fb86aa95c3d22d163d2ff46 Jul 28 16:21:02.120860 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:21:02.120825 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaeb9e49_9c13_4f71_9e6c_b14e68624005.slice/crio-f50f689467a3a1d8e59f6dd704ac397fee644d7cdbee286da1fcd85b6c97c70d WatchSource:0}: Error finding container f50f689467a3a1d8e59f6dd704ac397fee644d7cdbee286da1fcd85b6c97c70d: Status 404 returned error can't find the container with id f50f689467a3a1d8e59f6dd704ac397fee644d7cdbee286da1fcd85b6c97c70d Jul 28 16:21:02.121653 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:21:02.121451 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b3868a_68e2_408f_90a8_808c53e0aefe.slice/crio-26606383cc5a511203bf08415932e77f32d998077e32a250699c25895e377c32 WatchSource:0}: Error finding container 26606383cc5a511203bf08415932e77f32d998077e32a250699c25895e377c32: Status 404 returned error can't find the container with id 26606383cc5a511203bf08415932e77f32d998077e32a250699c25895e377c32 Jul 28 16:21:02.590479 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:02.590185 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64ljk\" (UniqueName: \"kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk\") pod \"network-check-target-gbdb4\" (UID: \"6d6d1243-b8d6-4402-96f1-5350263231ab\") " pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:02.590479 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:02.590437 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:21:02.590479 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:02.590470 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:21:02.590479 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:02.590486 2566 projected.go:196] Error preparing data for projected volume kube-api-access-64ljk for pod openshift-network-diagnostics/network-check-target-gbdb4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:21:02.590812 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:02.590465 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs\") pod \"network-metrics-daemon-gb4cw\" (UID: \"4a8f2962-0386-4066-a462-445cbb6c3f09\") " pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:02.590812 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:02.590546 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk podName:6d6d1243-b8d6-4402-96f1-5350263231ab nodeName:}" failed. No retries permitted until 2026-07-28 16:21:04.590524253 +0000 UTC m=+5.972713853 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-64ljk" (UniqueName: "kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk") pod "network-check-target-gbdb4" (UID: "6d6d1243-b8d6-4402-96f1-5350263231ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:21:02.590812 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:02.590554 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:21:02.590812 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:02.590611 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs podName:4a8f2962-0386-4066-a462-445cbb6c3f09 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:04.590593561 +0000 UTC m=+5.972783180 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs") pod "network-metrics-daemon-gb4cw" (UID: "4a8f2962-0386-4066-a462-445cbb6c3f09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:21:02.590812 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:02.590633 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret\") pod \"global-pull-secret-syncer-nzcls\" (UID: \"02ce611d-aef7-4996-bb0f-80279da441b5\") " pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:02.590812 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:02.590731 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:21:02.590812 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:02.590769 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret podName:02ce611d-aef7-4996-bb0f-80279da441b5 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:04.590757044 +0000 UTC m=+5.972946658 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret") pod "global-pull-secret-syncer-nzcls" (UID: "02ce611d-aef7-4996-bb0f-80279da441b5") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:21:02.859786 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:02.859685 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 16:21:03.017774 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:03.017694 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 16:16:01 +0000 UTC" deadline="2028-01-26 12:31:23.975818106 +0000 UTC" Jul 28 16:21:03.017774 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:03.017768 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13124h10m20.958055662s" Jul 28 16:21:03.104589 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:03.104554 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:03.104780 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:03.104718 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:03.148607 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:03.148508 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lhg5z" event={"ID":"312acc96-6bbf-4d5e-98fd-f5142efd2bdd","Type":"ContainerStarted","Data":"d1c7b421e4754fc50c28d140f99d42019cee89d00fbd22c9ac93e1588f4ddd31"} Jul 28 16:21:03.172637 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:03.172590 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-82p66" event={"ID":"4258901a-a0ea-48aa-921d-a3217ab8a119","Type":"ContainerStarted","Data":"c5d3ff5fbe5f0b2b5ec3959afd85f8a2ddbb71a61610cdeb80fad681f7e55b67"} Jul 28 16:21:03.177835 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:03.177730 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h54gg" event={"ID":"f6fa7f51-199b-4804-a481-e8641bad9f5b","Type":"ContainerStarted","Data":"2c61ed3fa89f0aa78b3b00d5cb49b9fe34ef83f86dbf69304d899017cd042ea9"} Jul 28 16:21:03.184543 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:03.184440 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-knhs8" event={"ID":"faeb9e49-9c13-4f71-9e6c-b14e68624005","Type":"ContainerStarted","Data":"f50f689467a3a1d8e59f6dd704ac397fee644d7cdbee286da1fcd85b6c97c70d"} Jul 28 16:21:03.191510 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:03.191411 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rvhjb" event={"ID":"c72d215a-f550-422a-98db-d21bedaca835","Type":"ContainerStarted","Data":"2716273100cdd2a9ea4919cb508d4f7a21be8e7a4ed0ad62aa7fcb0316362559"} Jul 28 16:21:03.201275 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:03.201240 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" event={"ID":"54b3868a-68e2-408f-90a8-808c53e0aefe","Type":"ContainerStarted","Data":"26606383cc5a511203bf08415932e77f32d998077e32a250699c25895e377c32"} Jul 28 16:21:03.204599 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:03.204561 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6jp82" event={"ID":"18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b","Type":"ContainerStarted","Data":"4993b2cea624756aa22f5b824ad5dabd4d22876d6fb86aa95c3d22d163d2ff46"} Jul 28 16:21:03.209591 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:03.209430 2566 generic.go:356] "Generic (PLEG): container finished" podID="e65469e9ab7d34cf69396aa7763d3b67" containerID="76723440e77535d452e992ef079830e4055aeba7abfc917ed9a11e9e8d2b1e3d" exitCode=0 Jul 28 16:21:03.209591 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:03.209528 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal" event={"ID":"e65469e9ab7d34cf69396aa7763d3b67","Type":"ContainerDied","Data":"76723440e77535d452e992ef079830e4055aeba7abfc917ed9a11e9e8d2b1e3d"} Jul 28 16:21:03.220592 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:03.220289 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-22.ec2.internal" event={"ID":"c8a16f2abd34aee43b0a0ab3f7f0846e","Type":"ContainerStarted","Data":"d9369139cb2174e8fdef358e337f45513c51b0b4efe71a3ee2144a5275f7fa36"} Jul 28 16:21:03.253814 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:03.253773 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" event={"ID":"ecee27e5-8dc3-4b95-a111-4cfa81feea7a","Type":"ContainerStarted","Data":"d2aa85d4754f3e34fe774b8cea9a8e8e90ef142ecb000aa2ae6c9d6eac4d083e"} Jul 28 16:21:03.270136 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:03.270093 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8fpkf" event={"ID":"4d2caa30-ece8-42cf-9dc4-e45e56dae4a2","Type":"ContainerStarted","Data":"ab153464b77d578028ff0391a946e8e123bb172795500ac3fe9722d0fe022a6c"} Jul 28 16:21:04.102386 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:04.102317 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:04.102927 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:04.102463 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:04.102927 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:04.102545 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:04.102927 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:04.102654 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:04.283892 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:04.283851 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal" event={"ID":"e65469e9ab7d34cf69396aa7763d3b67","Type":"ContainerStarted","Data":"2bddcb0b4d6523ee6b6bab98c2735ad4166e80ac777fb6598b134684a78f302e"} Jul 28 16:21:04.301343 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:04.300506 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-22.ec2.internal" podStartSLOduration=4.300481752 podStartE2EDuration="4.300481752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:03.242720629 +0000 UTC m=+4.624910256" watchObservedRunningTime="2026-07-28 16:21:04.300481752 +0000 UTC m=+5.682671377" Jul 28 16:21:04.607214 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:04.607146 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs\") pod \"network-metrics-daemon-gb4cw\" (UID: \"4a8f2962-0386-4066-a462-445cbb6c3f09\") " pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:04.607399 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:04.607219 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret\") pod \"global-pull-secret-syncer-nzcls\" (UID: \"02ce611d-aef7-4996-bb0f-80279da441b5\") " pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:04.607399 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:04.607253 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64ljk\" (UniqueName: \"kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk\") pod \"network-check-target-gbdb4\" (UID: \"6d6d1243-b8d6-4402-96f1-5350263231ab\") " pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:04.607516 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:04.607427 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:21:04.607516 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:04.607449 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:21:04.607516 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:04.607462 2566 projected.go:196] Error preparing data for projected volume kube-api-access-64ljk for pod openshift-network-diagnostics/network-check-target-gbdb4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:21:04.607665 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:04.607525 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk podName:6d6d1243-b8d6-4402-96f1-5350263231ab nodeName:}" failed. No retries permitted until 2026-07-28 16:21:08.6075051 +0000 UTC m=+9.989694715 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-64ljk" (UniqueName: "kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk") pod "network-check-target-gbdb4" (UID: "6d6d1243-b8d6-4402-96f1-5350263231ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:21:04.611752 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:04.607995 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:21:04.611752 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:04.608072 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs podName:4a8f2962-0386-4066-a462-445cbb6c3f09 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:08.608038505 +0000 UTC m=+9.990228123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs") pod "network-metrics-daemon-gb4cw" (UID: "4a8f2962-0386-4066-a462-445cbb6c3f09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:21:04.611752 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:04.608136 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:21:04.611752 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:04.608176 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret podName:02ce611d-aef7-4996-bb0f-80279da441b5 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:08.608161855 +0000 UTC m=+9.990351470 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret") pod "global-pull-secret-syncer-nzcls" (UID: "02ce611d-aef7-4996-bb0f-80279da441b5") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:21:05.103266 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:05.103180 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:05.103730 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:05.103352 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:06.101825 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:06.101786 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:06.102012 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:06.101971 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:06.102385 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:06.102359 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:06.102532 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:06.102442 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:07.104339 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:07.104276 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:07.104869 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:07.104426 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:08.102763 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:08.102094 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:08.102763 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:08.102238 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:08.102763 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:08.102309 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:08.102763 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:08.102387 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:08.641120 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:08.640150 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs\") pod \"network-metrics-daemon-gb4cw\" (UID: \"4a8f2962-0386-4066-a462-445cbb6c3f09\") " pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:08.641120 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:08.640227 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret\") pod \"global-pull-secret-syncer-nzcls\" (UID: \"02ce611d-aef7-4996-bb0f-80279da441b5\") " pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:08.641120 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:08.640267 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64ljk\" (UniqueName: \"kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk\") pod \"network-check-target-gbdb4\" (UID: \"6d6d1243-b8d6-4402-96f1-5350263231ab\") " pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:08.641120 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:08.640425 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:21:08.641120 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:08.640451 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:21:08.641120 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:08.640464 2566 projected.go:196] Error preparing data for projected volume kube-api-access-64ljk for pod openshift-network-diagnostics/network-check-target-gbdb4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:21:08.641120 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:08.640524 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk podName:6d6d1243-b8d6-4402-96f1-5350263231ab nodeName:}" failed. No retries permitted until 2026-07-28 16:21:16.64050505 +0000 UTC m=+18.022694666 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-64ljk" (UniqueName: "kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk") pod "network-check-target-gbdb4" (UID: "6d6d1243-b8d6-4402-96f1-5350263231ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:21:08.641120 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:08.640926 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:21:08.641120 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:08.640975 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs podName:4a8f2962-0386-4066-a462-445cbb6c3f09 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:16.640958727 +0000 UTC m=+18.023148341 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs") pod "network-metrics-daemon-gb4cw" (UID: "4a8f2962-0386-4066-a462-445cbb6c3f09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:21:08.641120 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:08.641032 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:21:08.641120 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:08.641085 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret podName:02ce611d-aef7-4996-bb0f-80279da441b5 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:16.641071625 +0000 UTC m=+18.023261226 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret") pod "global-pull-secret-syncer-nzcls" (UID: "02ce611d-aef7-4996-bb0f-80279da441b5") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:21:09.103330 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:09.103138 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:09.103330 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:09.103283 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:10.102529 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:10.102492 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:10.103097 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:10.102627 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:10.103097 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:10.103060 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:10.103227 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:10.103161 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:11.102270 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:11.102231 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:11.102465 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:11.102394 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:12.102073 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:12.102040 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:12.102507 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:12.102184 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:12.102507 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:12.102196 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:12.102507 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:12.102316 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:13.102660 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:13.102619 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:13.103114 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:13.102767 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:14.102347 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:14.102308 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:14.102347 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:14.102335 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:14.102596 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:14.102437 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:14.102596 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:14.102566 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:15.102702 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:15.102670 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:15.103150 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:15.102812 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:16.102174 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:16.102129 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:16.102350 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:16.102271 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:16.102350 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:16.102130 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:16.102452 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:16.102378 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:16.707869 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:16.707829 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret\") pod \"global-pull-secret-syncer-nzcls\" (UID: \"02ce611d-aef7-4996-bb0f-80279da441b5\") " pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:16.708370 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:16.707881 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64ljk\" (UniqueName: \"kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk\") pod \"network-check-target-gbdb4\" (UID: \"6d6d1243-b8d6-4402-96f1-5350263231ab\") " pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:16.708370 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:16.707916 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs\") pod \"network-metrics-daemon-gb4cw\" (UID: \"4a8f2962-0386-4066-a462-445cbb6c3f09\") " pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:16.708370 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:16.708016 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:21:16.708370 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:16.708016 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:21:16.708370 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:16.708056 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:21:16.708370 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:16.708081 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs podName:4a8f2962-0386-4066-a462-445cbb6c3f09 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:32.708062997 +0000 UTC m=+34.090252607 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs") pod "network-metrics-daemon-gb4cw" (UID: "4a8f2962-0386-4066-a462-445cbb6c3f09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:21:16.708370 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:16.708085 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:21:16.708370 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:16.708102 2566 projected.go:196] Error preparing data for projected volume kube-api-access-64ljk for pod openshift-network-diagnostics/network-check-target-gbdb4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:21:16.708370 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:16.708124 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret podName:02ce611d-aef7-4996-bb0f-80279da441b5 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:32.708097885 +0000 UTC m=+34.090287490 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret") pod "global-pull-secret-syncer-nzcls" (UID: "02ce611d-aef7-4996-bb0f-80279da441b5") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:21:16.708370 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:16.708152 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk podName:6d6d1243-b8d6-4402-96f1-5350263231ab nodeName:}" failed. No retries permitted until 2026-07-28 16:21:32.708137856 +0000 UTC m=+34.090327470 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-64ljk" (UniqueName: "kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk") pod "network-check-target-gbdb4" (UID: "6d6d1243-b8d6-4402-96f1-5350263231ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:21:17.102021 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:17.101984 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:17.102181 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:17.102136 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:18.102504 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:18.102469 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:18.102947 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:18.102473 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:18.102947 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:18.102582 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:18.102947 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:18.102688 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:19.103315 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:19.103277 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:19.103722 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:19.103409 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:20.102517 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:20.102480 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:20.102691 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:20.102480 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:20.102691 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:20.102633 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:20.102808 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:20.102730 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:21.101799 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:21.101768 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:21.102356 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:21.101891 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:22.101964 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.101594 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:22.102661 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.101670 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:22.102661 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:22.102082 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:22.102661 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:22.102124 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:22.326617 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.326575 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-82p66" event={"ID":"4258901a-a0ea-48aa-921d-a3217ab8a119","Type":"ContainerStarted","Data":"50ef52c2eabfc9707715d7b309e3cb1f23431c523c260ca330c0f7ca28e041b9"} Jul 28 16:21:22.328238 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.328208 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h54gg" event={"ID":"f6fa7f51-199b-4804-a481-e8641bad9f5b","Type":"ContainerStarted","Data":"ab6b16ac10ef341e3112161d020a484d8506e1951818ed8efbe0f03976d3b3d2"} Jul 28 16:21:22.329872 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.329839 2566 generic.go:356] "Generic (PLEG): container finished" podID="faeb9e49-9c13-4f71-9e6c-b14e68624005" containerID="1234df4b804b5cac4bbb135dab0ee5a377fee76916a12e24ef197dba0d1382ee" exitCode=0 Jul 28 16:21:22.330130 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.329890 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-knhs8" event={"ID":"faeb9e49-9c13-4f71-9e6c-b14e68624005","Type":"ContainerDied","Data":"1234df4b804b5cac4bbb135dab0ee5a377fee76916a12e24ef197dba0d1382ee"} Jul 28 16:21:22.333392 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.333377 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:21:22.333777 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.333753 2566 generic.go:356] "Generic (PLEG): container finished" podID="54b3868a-68e2-408f-90a8-808c53e0aefe" containerID="73f739204d17630dba1aa9bb06bebb089670641ed1768e6d54c05613a0aaf14f" exitCode=1 Jul 28 16:21:22.333874 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.333810 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" event={"ID":"54b3868a-68e2-408f-90a8-808c53e0aefe","Type":"ContainerStarted","Data":"c9a800ae47652b578967abbec2549749db6499b1e986943dbd83bb34c7b2a325"} Jul 28 16:21:22.333924 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.333878 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" event={"ID":"54b3868a-68e2-408f-90a8-808c53e0aefe","Type":"ContainerStarted","Data":"cbad7c20a8590e99e33394fdccacaa2d49d8ad4934653369f5ee94f116988254"} Jul 28 16:21:22.333960 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.333932 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" event={"ID":"54b3868a-68e2-408f-90a8-808c53e0aefe","Type":"ContainerStarted","Data":"58cc230ba917edb817637d22e448845f548e45ddc5adf83c0099b91c48ae9b7b"} Jul 28 16:21:22.334027 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.333996 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" event={"ID":"54b3868a-68e2-408f-90a8-808c53e0aefe","Type":"ContainerStarted","Data":"7ba741941d8b019c735743bf49ec450d87e62d2830f5487d2c9fc44d7c24475e"} Jul 28 16:21:22.334067 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.334043 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" event={"ID":"54b3868a-68e2-408f-90a8-808c53e0aefe","Type":"ContainerDied","Data":"73f739204d17630dba1aa9bb06bebb089670641ed1768e6d54c05613a0aaf14f"} Jul 28 16:21:22.334115 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.334091 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" event={"ID":"54b3868a-68e2-408f-90a8-808c53e0aefe","Type":"ContainerStarted","Data":"50de0f19824b3354fef3047ce92ea6c8fc3fe26b934cf7eb099355f59f8e8cbf"} Jul 28 16:21:22.335337 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.335316 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6jp82" event={"ID":"18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b","Type":"ContainerStarted","Data":"573387f9002d5d587438f161466599d4ce15bf051682808e6f7d85ec470e84d7"} Jul 28 16:21:22.336793 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.336773 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" event={"ID":"ecee27e5-8dc3-4b95-a111-4cfa81feea7a","Type":"ContainerStarted","Data":"55e5ce890ebaee34003ae20b087912215469441f2241ce7940de0f638bc2e01f"} Jul 28 16:21:22.338328 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.338305 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8fpkf" event={"ID":"4d2caa30-ece8-42cf-9dc4-e45e56dae4a2","Type":"ContainerStarted","Data":"eca44b9d467a3b6c675c2c34a987d660fe5634ec0b029dfc88fac3599b1259f6"} Jul 28 16:21:22.339898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.339875 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lhg5z" event={"ID":"312acc96-6bbf-4d5e-98fd-f5142efd2bdd","Type":"ContainerStarted","Data":"5424c078dc370b82e72215d8e5588b77221bc81e73b8d4ed14b6bfb8cc911b9a"} Jul 28 16:21:22.344584 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.344535 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-82p66" podStartSLOduration=3.136528733 podStartE2EDuration="22.344524842s" totalImagesPullingTime="19.207996109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:22.343380923 +0000 UTC m=+23.725570541" watchObservedRunningTime="2026-07-28 16:21:22.344524842 +0000 UTC m=+23.726714506" Jul 28 16:21:22.344707 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.344672 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-22.ec2.internal" podStartSLOduration=22.34466708 podStartE2EDuration="22.34466708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:04.301291451 +0000 UTC m=+5.683481074" watchObservedRunningTime="2026-07-28 16:21:22.34466708 +0000 UTC m=+23.726856698" Jul 28 16:21:22.357823 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.357767 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8fpkf" podStartSLOduration=3.133472372 podStartE2EDuration="22.357747158s" totalImagesPullingTime="19.224274786s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:22.357059719 +0000 UTC m=+23.739249336" watchObservedRunningTime="2026-07-28 16:21:22.357747158 +0000 UTC m=+23.739936776" Jul 28 16:21:22.377871 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.377767 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6jp82" podStartSLOduration=3.161695513 podStartE2EDuration="22.377748064s" totalImagesPullingTime="19.216052551s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:22.377636139 +0000 UTC m=+23.759825756" watchObservedRunningTime="2026-07-28 16:21:22.377748064 +0000 UTC m=+23.759937682" Jul 28 16:21:22.419617 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.419559 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lhg5z" podStartSLOduration=3.1969717539999998 podStartE2EDuration="22.419537706s" totalImagesPullingTime="19.222565952s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:22.418943606 +0000 UTC m=+23.801133223" watchObservedRunningTime="2026-07-28 16:21:22.419537706 +0000 UTC m=+23.801727341" Jul 28 16:21:22.437393 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.437345 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-h54gg" podStartSLOduration=3.229284383 podStartE2EDuration="22.437329386s" totalImagesPullingTime="19.208045003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:22.437123772 +0000 UTC m=+23.819313400" watchObservedRunningTime="2026-07-28 16:21:22.437329386 +0000 UTC m=+23.819519003" Jul 28 16:21:22.902384 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:22.902346 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 28 16:21:23.038134 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:23.037965 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T16:21:22.90236453Z","UUID":"f262f22d-8099-4cf9-baad-ea340cb8008b","Handler":null,"Name":"","Endpoint":""} Jul 28 16:21:23.039533 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:23.039506 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 28 16:21:23.039634 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:23.039541 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 28 16:21:23.102471 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:23.102433 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:23.102901 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:23.102593 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:23.224410 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:23.224380 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-h54gg" Jul 28 16:21:23.224985 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:23.224951 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-h54gg" Jul 28 16:21:23.344235 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:23.344146 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" event={"ID":"ecee27e5-8dc3-4b95-a111-4cfa81feea7a","Type":"ContainerStarted","Data":"dd0eaa48a4c2ed79da8eaf693c873153e80a601dd55ba282d1f024258ae6dc61"} Jul 28 16:21:23.345750 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:23.345691 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rvhjb" event={"ID":"c72d215a-f550-422a-98db-d21bedaca835","Type":"ContainerStarted","Data":"df0084bdb1b83b7d0ac036d2663969ce2625a1b5967ed19c1a290cf4b6ff24e6"} Jul 28 16:21:23.346555 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:23.346531 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-h54gg" Jul 28 16:21:23.346962 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:23.346943 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-h54gg" Jul 28 16:21:23.374028 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:23.373954 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rvhjb" podStartSLOduration=4.152077864 podStartE2EDuration="23.373933708s" totalImagesPullingTime="19.221855844s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:23.359248188 +0000 UTC m=+24.741437805" watchObservedRunningTime="2026-07-28 16:21:23.373933708 +0000 UTC m=+24.756123327" Jul 28 16:21:24.102081 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:24.102049 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:24.102247 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:24.102090 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:24.102247 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:24.102174 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:24.102370 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:24.102298 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:24.350258 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:24.350211 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" event={"ID":"ecee27e5-8dc3-4b95-a111-4cfa81feea7a","Type":"ContainerStarted","Data":"bb09c6057f360b3fc0b03f330ec1d61d4820f6ca5f7ba1acaff1f9fca795b2a0"} Jul 28 16:21:24.354117 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:24.354092 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:21:24.354704 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:24.354672 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" event={"ID":"54b3868a-68e2-408f-90a8-808c53e0aefe","Type":"ContainerStarted","Data":"1051aa397029438ec7293e49606429841af49074b2b274bbac9ca213ec42481b"} Jul 28 16:21:24.371024 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:24.370948 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fmxvl" podStartSLOduration=4.007671384 podStartE2EDuration="25.370931557s" totalImagesPullingTime="21.363260173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:20:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:24.370378276 +0000 UTC m=+25.752567905" watchObservedRunningTime="2026-07-28 16:21:24.370931557 +0000 UTC m=+25.753121175" Jul 28 16:21:25.105295 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:25.105266 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:25.105465 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:25.105378 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:26.102616 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:26.102429 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:26.103124 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:26.102436 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:26.103124 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:26.102716 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:26.103124 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:26.102785 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:27.105422 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:27.105393 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:27.106103 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:27.105494 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:27.362451 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:27.362369 2566 generic.go:356] "Generic (PLEG): container finished" podID="faeb9e49-9c13-4f71-9e6c-b14e68624005" containerID="ee19fb5f6731b563cc734288413f3dba966e59aee76c35c99113e959359a47ec" exitCode=0 Jul 28 16:21:27.362600 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:27.362455 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-knhs8" event={"ID":"faeb9e49-9c13-4f71-9e6c-b14e68624005","Type":"ContainerDied","Data":"ee19fb5f6731b563cc734288413f3dba966e59aee76c35c99113e959359a47ec"} Jul 28 16:21:27.365888 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:27.365873 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:21:27.366228 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:27.366209 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" event={"ID":"54b3868a-68e2-408f-90a8-808c53e0aefe","Type":"ContainerStarted","Data":"b3e011864043cc787ad89edd2402d1bdebe0e025b832f2a5cf5cd9795d624e4b"} Jul 28 16:21:27.366474 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:27.366450 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:27.366530 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:27.366519 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:27.366658 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:27.366642 2566 scope.go:117] "RemoveContainer" containerID="73f739204d17630dba1aa9bb06bebb089670641ed1768e6d54c05613a0aaf14f" Jul 28 16:21:27.387579 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:27.387553 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:28.102590 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:28.102406 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:28.102749 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:28.102435 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:28.102749 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:28.102682 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:28.102857 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:28.102782 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:28.371197 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:28.371113 2566 generic.go:356] "Generic (PLEG): container finished" podID="faeb9e49-9c13-4f71-9e6c-b14e68624005" containerID="d51ff620852bd32880742c1ab6d8a00f72c1a08560a952d283950897f0005175" exitCode=0 Jul 28 16:21:28.371575 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:28.371234 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-knhs8" event={"ID":"faeb9e49-9c13-4f71-9e6c-b14e68624005","Type":"ContainerDied","Data":"d51ff620852bd32880742c1ab6d8a00f72c1a08560a952d283950897f0005175"} Jul 28 16:21:28.375429 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:28.375403 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:21:28.375838 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:28.375809 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" event={"ID":"54b3868a-68e2-408f-90a8-808c53e0aefe","Type":"ContainerStarted","Data":"231c38560659b842cf87a1753b69308137a0aba738444c9baf7b3bd525fc8ad1"} Jul 28 16:21:28.376287 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:28.376265 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:28.391765 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:28.391741 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:21:28.419228 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:28.419170 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" podStartSLOduration=9.963212924 podStartE2EDuration="29.419154746s" totalImagesPullingTime="19.455941822s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:20:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:28.417746279 +0000 UTC m=+29.799935895" watchObservedRunningTime="2026-07-28 16:21:28.419154746 +0000 UTC m=+29.801344357" Jul 28 16:21:28.567207 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:28.567171 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nzcls"] Jul 28 16:21:28.567351 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:28.567339 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:28.567499 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:28.567474 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:28.570490 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:28.570462 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gbdb4"] Jul 28 16:21:28.570614 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:28.570569 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:28.570685 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:28.570668 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:28.571174 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:28.571146 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gb4cw"] Jul 28 16:21:28.571264 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:28.571253 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:28.571351 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:28.571335 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:29.380727 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:29.380692 2566 generic.go:356] "Generic (PLEG): container finished" podID="faeb9e49-9c13-4f71-9e6c-b14e68624005" containerID="f882fdc8407f3707d56c5299a4aa5588c57f7c359a24893612b601668ae44132" exitCode=0 Jul 28 16:21:29.381133 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:29.380775 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-knhs8" event={"ID":"faeb9e49-9c13-4f71-9e6c-b14e68624005","Type":"ContainerDied","Data":"f882fdc8407f3707d56c5299a4aa5588c57f7c359a24893612b601668ae44132"} Jul 28 16:21:30.101643 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:30.101606 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:30.101643 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:30.101633 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:30.101881 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:30.101745 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:30.101881 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:30.101766 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:30.101881 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:30.101840 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:30.102045 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:30.101946 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:32.101726 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:32.101686 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:32.102098 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:32.101740 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:32.102098 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:32.101783 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:32.102098 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:32.101798 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:32.102098 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:32.101866 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:32.102098 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:32.101934 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:32.728344 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:32.728303 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64ljk\" (UniqueName: \"kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk\") pod \"network-check-target-gbdb4\" (UID: \"6d6d1243-b8d6-4402-96f1-5350263231ab\") " pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:32.728523 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:32.728368 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs\") pod \"network-metrics-daemon-gb4cw\" (UID: \"4a8f2962-0386-4066-a462-445cbb6c3f09\") " pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:32.728523 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:32.728416 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret\") pod \"global-pull-secret-syncer-nzcls\" (UID: \"02ce611d-aef7-4996-bb0f-80279da441b5\") " pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:32.728523 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:32.728481 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:21:32.728523 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:32.728505 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:21:32.728523 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:32.728521 2566 projected.go:196] Error preparing data for projected volume kube-api-access-64ljk for pod openshift-network-diagnostics/network-check-target-gbdb4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:21:32.728523 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:32.728517 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:21:32.728808 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:32.728527 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:21:32.728808 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:32.728568 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk podName:6d6d1243-b8d6-4402-96f1-5350263231ab nodeName:}" failed. No retries permitted until 2026-07-28 16:22:04.728555292 +0000 UTC m=+66.110744887 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-64ljk" (UniqueName: "kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk") pod "network-check-target-gbdb4" (UID: "6d6d1243-b8d6-4402-96f1-5350263231ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:21:32.728808 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:32.728583 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret podName:02ce611d-aef7-4996-bb0f-80279da441b5 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:04.728576516 +0000 UTC m=+66.110766112 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret") pod "global-pull-secret-syncer-nzcls" (UID: "02ce611d-aef7-4996-bb0f-80279da441b5") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:21:32.728808 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:32.728595 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs podName:4a8f2962-0386-4066-a462-445cbb6c3f09 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:04.728588286 +0000 UTC m=+66.110777881 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs") pod "network-metrics-daemon-gb4cw" (UID: "4a8f2962-0386-4066-a462-445cbb6c3f09") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:21:34.102429 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.102231 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:34.102888 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.102283 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:34.102888 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:34.102470 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzcls" podUID="02ce611d-aef7-4996-bb0f-80279da441b5" Jul 28 16:21:34.102888 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.102290 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:34.102888 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:34.102547 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gb4cw" podUID="4a8f2962-0386-4066-a462-445cbb6c3f09" Jul 28 16:21:34.102888 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:34.102630 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gbdb4" podUID="6d6d1243-b8d6-4402-96f1-5350263231ab" Jul 28 16:21:34.396471 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.396389 2566 generic.go:356] "Generic (PLEG): container finished" podID="faeb9e49-9c13-4f71-9e6c-b14e68624005" containerID="b8733ea4fb6ab6fbf71e6152bfc9ef604312f68403187260836c29f7704077e2" exitCode=0 Jul 28 16:21:34.396471 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.396444 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-knhs8" event={"ID":"faeb9e49-9c13-4f71-9e6c-b14e68624005","Type":"ContainerDied","Data":"b8733ea4fb6ab6fbf71e6152bfc9ef604312f68403187260836c29f7704077e2"} Jul 28 16:21:34.488260 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.488236 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-22.ec2.internal" event="NodeReady" Jul 28 16:21:34.488398 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.488364 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 28 16:21:34.519607 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.519582 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-554f97cd94-4klh7"] Jul 28 16:21:34.544863 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.544828 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fdhkh"] Jul 28 16:21:34.545023 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.545002 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.547677 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.547653 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 28 16:21:34.547809 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.547691 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 28 16:21:34.548127 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.548107 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 28 16:21:34.549014 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.548354 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zshnb\"" Jul 28 16:21:34.554126 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.554097 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 28 16:21:34.564109 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.564087 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-554f97cd94-4klh7"] Jul 28 16:21:34.564222 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.564123 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tftts"] Jul 28 16:21:34.564222 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.564163 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:34.566584 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.566561 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 28 16:21:34.566584 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.566583 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 28 16:21:34.566755 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.566607 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hmx4q\"" Jul 28 16:21:34.584439 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.584419 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fdhkh"] Jul 28 16:21:34.584547 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.584453 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tftts"] Jul 28 16:21:34.584601 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.584555 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tftts" Jul 28 16:21:34.586994 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.586950 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 28 16:21:34.587226 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.587209 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-sllng\"" Jul 28 16:21:34.587319 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.587245 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 28 16:21:34.587319 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.587283 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 28 16:21:34.642175 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.642153 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:34.642280 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.642181 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-bound-sa-token\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.642280 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.642212 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-tmp-dir\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:34.642280 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.642226 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a43fcfe-07e8-4f38-bfff-fb4cdedb6d3b-cert\") pod \"ingress-canary-tftts\" (UID: \"7a43fcfe-07e8-4f38-bfff-fb4cdedb6d3b\") " pod="openshift-ingress-canary/ingress-canary-tftts" Jul 28 16:21:34.642280 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.642260 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-ca-trust-extracted\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.642405 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.642284 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj87z\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-kube-api-access-tj87z\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.642405 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.642303 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-config-volume\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:34.642405 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.642318 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kngdr\" (UniqueName: \"kubernetes.io/projected/7a43fcfe-07e8-4f38-bfff-fb4cdedb6d3b-kube-api-access-kngdr\") pod \"ingress-canary-tftts\" (UID: \"7a43fcfe-07e8-4f38-bfff-fb4cdedb6d3b\") " pod="openshift-ingress-canary/ingress-canary-tftts" Jul 28 16:21:34.642405 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.642359 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.642405 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.642391 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-trusted-ca\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.642545 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.642449 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-certificates\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.642545 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.642474 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-installation-pull-secrets\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.642545 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.642500 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbbmk\" (UniqueName: \"kubernetes.io/projected/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-kube-api-access-jbbmk\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:34.642545 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.642520 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-image-registry-private-configuration\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.743641 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.743610 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:34.743807 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.743647 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-bound-sa-token\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.743807 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.743688 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-tmp-dir\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:34.743807 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.743708 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a43fcfe-07e8-4f38-bfff-fb4cdedb6d3b-cert\") pod \"ingress-canary-tftts\" (UID: \"7a43fcfe-07e8-4f38-bfff-fb4cdedb6d3b\") " pod="openshift-ingress-canary/ingress-canary-tftts" Jul 28 16:21:34.743807 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.743745 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-ca-trust-extracted\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.743807 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:34.743767 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 16:21:34.744112 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:34.743845 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls podName:08c6af7a-3f2a-491b-aee3-61a1dafe2c0a nodeName:}" failed. No retries permitted until 2026-07-28 16:21:35.243824859 +0000 UTC m=+36.626014479 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls") pod "dns-default-fdhkh" (UID: "08c6af7a-3f2a-491b-aee3-61a1dafe2c0a") : secret "dns-default-metrics-tls" not found Jul 28 16:21:34.744112 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.743770 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj87z\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-kube-api-access-tj87z\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.744112 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.743952 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-config-volume\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:34.744112 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.744011 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kngdr\" (UniqueName: \"kubernetes.io/projected/7a43fcfe-07e8-4f38-bfff-fb4cdedb6d3b-kube-api-access-kngdr\") pod \"ingress-canary-tftts\" (UID: \"7a43fcfe-07e8-4f38-bfff-fb4cdedb6d3b\") " pod="openshift-ingress-canary/ingress-canary-tftts" Jul 28 16:21:34.744112 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.744036 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.744112 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.744058 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-trusted-ca\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.744427 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.744138 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-tmp-dir\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:34.744427 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:34.744163 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 16:21:34.744427 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:34.744174 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-554f97cd94-4klh7: secret "image-registry-tls" not found Jul 28 16:21:34.744427 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.744189 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-ca-trust-extracted\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.744427 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:34.744226 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls podName:bae9c53c-7ff2-43b0-b0ac-5433a37338e4 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:35.244213318 +0000 UTC m=+36.626402915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls") pod "image-registry-554f97cd94-4klh7" (UID: "bae9c53c-7ff2-43b0-b0ac-5433a37338e4") : secret "image-registry-tls" not found Jul 28 16:21:34.744427 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.744252 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-certificates\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.744427 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.744278 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-installation-pull-secrets\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.744427 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.744385 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbbmk\" (UniqueName: \"kubernetes.io/projected/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-kube-api-access-jbbmk\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:34.744427 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.744424 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-image-registry-private-configuration\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.744877 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.744538 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-config-volume\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:34.744877 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.744789 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-certificates\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.745044 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.745021 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-trusted-ca\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.748140 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.748120 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-image-registry-private-configuration\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.748225 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.748120 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a43fcfe-07e8-4f38-bfff-fb4cdedb6d3b-cert\") pod \"ingress-canary-tftts\" (UID: \"7a43fcfe-07e8-4f38-bfff-fb4cdedb6d3b\") " pod="openshift-ingress-canary/ingress-canary-tftts" Jul 28 16:21:34.748225 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.748195 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-installation-pull-secrets\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.755138 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.755107 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbbmk\" (UniqueName: \"kubernetes.io/projected/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-kube-api-access-jbbmk\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:34.755330 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.755308 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-bound-sa-token\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.755420 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.755392 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj87z\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-kube-api-access-tj87z\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:34.755481 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.755467 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kngdr\" (UniqueName: \"kubernetes.io/projected/7a43fcfe-07e8-4f38-bfff-fb4cdedb6d3b-kube-api-access-kngdr\") pod \"ingress-canary-tftts\" (UID: \"7a43fcfe-07e8-4f38-bfff-fb4cdedb6d3b\") " pod="openshift-ingress-canary/ingress-canary-tftts" Jul 28 16:21:34.948685 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:34.948650 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tftts" Jul 28 16:21:35.138463 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:35.138436 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tftts"] Jul 28 16:21:35.141698 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:21:35.141673 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a43fcfe_07e8_4f38_bfff_fb4cdedb6d3b.slice/crio-95386bcc0de1a0ce8cc391061fc0e1d4b7506d531067319e46bb4795007a5e35 WatchSource:0}: Error finding container 95386bcc0de1a0ce8cc391061fc0e1d4b7506d531067319e46bb4795007a5e35: Status 404 returned error can't find the container with id 95386bcc0de1a0ce8cc391061fc0e1d4b7506d531067319e46bb4795007a5e35 Jul 28 16:21:35.248960 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:35.248787 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:35.249156 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:35.248968 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 16:21:35.249156 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:35.249013 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:35.249156 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:35.249060 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls podName:08c6af7a-3f2a-491b-aee3-61a1dafe2c0a nodeName:}" failed. No retries permitted until 2026-07-28 16:21:36.24903812 +0000 UTC m=+37.631227716 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls") pod "dns-default-fdhkh" (UID: "08c6af7a-3f2a-491b-aee3-61a1dafe2c0a") : secret "dns-default-metrics-tls" not found Jul 28 16:21:35.249156 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:35.249105 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 16:21:35.249156 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:35.249115 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-554f97cd94-4klh7: secret "image-registry-tls" not found Jul 28 16:21:35.249156 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:35.249153 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls podName:bae9c53c-7ff2-43b0-b0ac-5433a37338e4 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:36.249142313 +0000 UTC m=+37.631331909 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls") pod "image-registry-554f97cd94-4klh7" (UID: "bae9c53c-7ff2-43b0-b0ac-5433a37338e4") : secret "image-registry-tls" not found Jul 28 16:21:35.401603 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:35.401522 2566 generic.go:356] "Generic (PLEG): container finished" podID="faeb9e49-9c13-4f71-9e6c-b14e68624005" containerID="80735ef25f5094c0ab00c873bd39211dd997e4952530d934981dae7a277f720e" exitCode=0 Jul 28 16:21:35.401766 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:35.401604 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-knhs8" event={"ID":"faeb9e49-9c13-4f71-9e6c-b14e68624005","Type":"ContainerDied","Data":"80735ef25f5094c0ab00c873bd39211dd997e4952530d934981dae7a277f720e"} Jul 28 16:21:35.402690 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:35.402666 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tftts" event={"ID":"7a43fcfe-07e8-4f38-bfff-fb4cdedb6d3b","Type":"ContainerStarted","Data":"95386bcc0de1a0ce8cc391061fc0e1d4b7506d531067319e46bb4795007a5e35"} Jul 28 16:21:36.102229 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:36.102191 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:21:36.102406 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:36.102192 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:21:36.102406 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:36.102314 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:21:36.105136 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:36.105115 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 16:21:36.105136 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:36.105126 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 16:21:36.105307 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:36.105103 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 16:21:36.106238 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:36.106216 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 16:21:36.106357 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:36.106241 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nr6mp\"" Jul 28 16:21:36.106357 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:36.106322 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gnm96\"" Jul 28 16:21:36.258071 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:36.258039 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:36.258511 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:36.258186 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 16:21:36.258511 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:36.258282 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:36.258511 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:36.258415 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 16:21:36.258511 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:36.258430 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-554f97cd94-4klh7: secret "image-registry-tls" not found Jul 28 16:21:36.258511 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:36.258478 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls podName:bae9c53c-7ff2-43b0-b0ac-5433a37338e4 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:38.258459428 +0000 UTC m=+39.640649041 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls") pod "image-registry-554f97cd94-4klh7" (UID: "bae9c53c-7ff2-43b0-b0ac-5433a37338e4") : secret "image-registry-tls" not found Jul 28 16:21:36.258511 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:36.258495 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls podName:08c6af7a-3f2a-491b-aee3-61a1dafe2c0a nodeName:}" failed. No retries permitted until 2026-07-28 16:21:38.258487312 +0000 UTC m=+39.640676909 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls") pod "dns-default-fdhkh" (UID: "08c6af7a-3f2a-491b-aee3-61a1dafe2c0a") : secret "dns-default-metrics-tls" not found Jul 28 16:21:36.407913 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:36.407836 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-knhs8" event={"ID":"faeb9e49-9c13-4f71-9e6c-b14e68624005","Type":"ContainerStarted","Data":"025e44e5cde02a3294f8cf2334b0d4e9548cbd22d4a8a5d6e2ef4df59cda21ef"} Jul 28 16:21:36.430957 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:36.430911 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-knhs8" podStartSLOduration=7.264931277 podStartE2EDuration="36.430896848s" totalImagesPullingTime="29.165965571s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:36.429063089 +0000 UTC m=+37.811252710" watchObservedRunningTime="2026-07-28 16:21:36.430896848 +0000 UTC m=+37.813086465" Jul 28 16:21:36.580377 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:36.580327 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-h54gg_f6fa7f51-199b-4804-a481-e8641bad9f5b/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:36.670741 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:36.670661 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-h54gg_f6fa7f51-199b-4804-a481-e8641bad9f5b/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:38.274767 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:38.274713 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:38.275132 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:38.274798 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:38.275132 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:38.274867 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 16:21:38.275132 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:38.274896 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 16:21:38.275132 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:38.274906 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-554f97cd94-4klh7: secret "image-registry-tls" not found Jul 28 16:21:38.275132 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:38.274934 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls podName:08c6af7a-3f2a-491b-aee3-61a1dafe2c0a nodeName:}" failed. No retries permitted until 2026-07-28 16:21:42.274917708 +0000 UTC m=+43.657107307 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls") pod "dns-default-fdhkh" (UID: "08c6af7a-3f2a-491b-aee3-61a1dafe2c0a") : secret "dns-default-metrics-tls" not found Jul 28 16:21:38.275132 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:38.274949 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls podName:bae9c53c-7ff2-43b0-b0ac-5433a37338e4 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:42.274941826 +0000 UTC m=+43.657131426 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls") pod "image-registry-554f97cd94-4klh7" (UID: "bae9c53c-7ff2-43b0-b0ac-5433a37338e4") : secret "image-registry-tls" not found Jul 28 16:21:38.415310 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:38.415184 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tftts" event={"ID":"7a43fcfe-07e8-4f38-bfff-fb4cdedb6d3b","Type":"ContainerStarted","Data":"d71ba4ee4d3b169fe7ac5d502fbdf8446d2576f49e9e7b3e28c550b1cedf5a4f"} Jul 28 16:21:38.430903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:38.430854 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tftts" podStartSLOduration=1.8024327740000001 podStartE2EDuration="4.430839249s" totalImagesPullingTime="2.628406475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:38.430216453 +0000 UTC m=+39.812406065" watchObservedRunningTime="2026-07-28 16:21:38.430839249 +0000 UTC m=+39.813028866" Jul 28 16:21:39.744469 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:39.744265 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-h54gg_f6fa7f51-199b-4804-a481-e8641bad9f5b/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:39.844495 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:39.844461 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-h54gg_f6fa7f51-199b-4804-a481-e8641bad9f5b/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:42.303631 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:42.303596 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:42.304061 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:42.303651 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:42.304061 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:42.303758 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 16:21:42.304061 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:42.303817 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 16:21:42.304061 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:42.303827 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-554f97cd94-4klh7: secret "image-registry-tls" not found Jul 28 16:21:42.304061 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:42.303839 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls podName:08c6af7a-3f2a-491b-aee3-61a1dafe2c0a nodeName:}" failed. No retries permitted until 2026-07-28 16:21:50.303821061 +0000 UTC m=+51.686010657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls") pod "dns-default-fdhkh" (UID: "08c6af7a-3f2a-491b-aee3-61a1dafe2c0a") : secret "dns-default-metrics-tls" not found Jul 28 16:21:42.304061 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:42.303864 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls podName:bae9c53c-7ff2-43b0-b0ac-5433a37338e4 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:50.303852789 +0000 UTC m=+51.686042386 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls") pod "image-registry-554f97cd94-4klh7" (UID: "bae9c53c-7ff2-43b0-b0ac-5433a37338e4") : secret "image-registry-tls" not found Jul 28 16:21:50.361756 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:50.361715 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:21:50.362274 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:50.361792 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:21:50.362274 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:50.361875 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 16:21:50.362274 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:50.361900 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 16:21:50.362274 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:50.361911 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-554f97cd94-4klh7: secret "image-registry-tls" not found Jul 28 16:21:50.362274 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:50.361948 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls podName:08c6af7a-3f2a-491b-aee3-61a1dafe2c0a nodeName:}" failed. No retries permitted until 2026-07-28 16:22:06.361928054 +0000 UTC m=+67.744117664 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls") pod "dns-default-fdhkh" (UID: "08c6af7a-3f2a-491b-aee3-61a1dafe2c0a") : secret "dns-default-metrics-tls" not found Jul 28 16:21:50.362274 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:50.361963 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls podName:bae9c53c-7ff2-43b0-b0ac-5433a37338e4 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:06.36195725 +0000 UTC m=+67.744146849 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls") pod "image-registry-554f97cd94-4klh7" (UID: "bae9c53c-7ff2-43b0-b0ac-5433a37338e4") : secret "image-registry-tls" not found Jul 28 16:21:58.151123 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.151091 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2m7fh"] Jul 28 16:21:58.166916 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.166871 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-9hwqx"] Jul 28 16:21:58.167080 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.166928 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2m7fh" Jul 28 16:21:58.169618 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.169596 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 28 16:21:58.169618 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.169605 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 28 16:21:58.169795 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.169605 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-8qp77\"" Jul 28 16:21:58.179030 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.179007 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2m7fh"] Jul 28 16:21:58.179141 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.179047 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-9hwqx"] Jul 28 16:21:58.179141 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.179067 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.181533 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.181512 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-b9n9l\"" Jul 28 16:21:58.181668 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.181510 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 28 16:21:58.181668 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.181594 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 28 16:21:58.181801 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.181765 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 28 16:21:58.182479 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.182464 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 28 16:21:58.187210 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.187193 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 28 16:21:58.255044 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.255013 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n"] Jul 28 16:21:58.276516 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.276490 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt"] Jul 28 16:21:58.276656 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.276611 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n" Jul 28 16:21:58.279131 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.279108 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 28 16:21:58.279131 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.279119 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 28 16:21:58.279362 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.279168 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 28 16:21:58.279492 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.279479 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 28 16:21:58.279545 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.279481 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-42zzj\"" Jul 28 16:21:58.296205 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.296177 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-855657b557-78fr5"] Jul 28 16:21:58.296338 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.296257 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:21:58.299285 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.299263 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 28 16:21:58.300469 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.300452 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 28 16:21:58.300559 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.300452 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dpb5t\"" Jul 28 16:21:58.300559 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.300453 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 28 16:21:58.300559 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.300514 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 28 16:21:58.316281 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.316253 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7"] Jul 28 16:21:58.316429 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.316376 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:21:58.318950 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.318934 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 28 16:21:58.319058 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.319006 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 28 16:21:58.319401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.319386 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 28 16:21:58.319614 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.319600 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 28 16:21:58.320320 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.320303 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 28 16:21:58.320524 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.320507 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 28 16:21:58.320595 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.320565 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j7dqr\"" Jul 28 16:21:58.325367 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.325349 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/676a1e19-6f6c-42a4-881c-730a584c1c2b-serving-cert\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.325463 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.325378 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/676a1e19-6f6c-42a4-881c-730a584c1c2b-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.325463 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.325417 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/676a1e19-6f6c-42a4-881c-730a584c1c2b-tmp\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.325626 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.325496 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/676a1e19-6f6c-42a4-881c-730a584c1c2b-snapshots\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.325626 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.325516 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/676a1e19-6f6c-42a4-881c-730a584c1c2b-service-ca-bundle\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.325626 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.325602 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqktx\" (UniqueName: \"kubernetes.io/projected/ce70be22-819d-4675-9c98-380ffd733f4c-kube-api-access-hqktx\") pod \"volume-data-source-validator-549c6d54cf-2m7fh\" (UID: \"ce70be22-819d-4675-9c98-380ffd733f4c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2m7fh" Jul 28 16:21:58.325746 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.325636 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8842\" (UniqueName: \"kubernetes.io/projected/676a1e19-6f6c-42a4-881c-730a584c1c2b-kube-api-access-g8842\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.341479 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.341454 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n"] Jul 28 16:21:58.341574 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.341485 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7"] Jul 28 16:21:58.341574 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.341502 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt"] Jul 28 16:21:58.341574 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.341516 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-855657b557-78fr5"] Jul 28 16:21:58.341574 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.341565 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" Jul 28 16:21:58.344030 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.343963 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 28 16:21:58.344443 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.344427 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 28 16:21:58.344526 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.344430 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-f6p2c\"" Jul 28 16:21:58.344526 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.344453 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 28 16:21:58.354614 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.354595 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-fcmgz"] Jul 28 16:21:58.375418 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.375389 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-fcmgz"] Jul 28 16:21:58.375550 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.375482 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:21:58.378175 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.378149 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 28 16:21:58.378656 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.378620 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 28 16:21:58.378656 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.378634 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2q2dj\"" Jul 28 16:21:58.378959 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.378942 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 28 16:21:58.379606 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.379577 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 28 16:21:58.390822 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.390793 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 28 16:21:58.426880 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.426794 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/676a1e19-6f6c-42a4-881c-730a584c1c2b-tmp\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.426880 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.426833 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:21:58.426880 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.426853 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8842\" (UniqueName: \"kubernetes.io/projected/676a1e19-6f6c-42a4-881c-730a584c1c2b-kube-api-access-g8842\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.426880 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.426876 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fz4bt\" (UID: \"54e904f0-e7ab-4882-9b0d-a91248f578a4\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:21:58.427152 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.426933 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-default-certificate\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:21:58.427152 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.426950 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6bn7\" (UniqueName: \"kubernetes.io/projected/cab67026-e646-4a71-9c1a-e21d1fda8536-kube-api-access-g6bn7\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:21:58.427152 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.426987 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/676a1e19-6f6c-42a4-881c-730a584c1c2b-snapshots\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.427152 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.427053 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhblr\" (UniqueName: \"kubernetes.io/projected/54e904f0-e7ab-4882-9b0d-a91248f578a4-kube-api-access-jhblr\") pod \"cluster-monitoring-operator-548b489cb8-fz4bt\" (UID: \"54e904f0-e7ab-4882-9b0d-a91248f578a4\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:21:58.427152 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.427129 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqwcs\" (UniqueName: \"kubernetes.io/projected/51fb28cb-f191-418e-ae54-43ca310712ec-kube-api-access-gqwcs\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n\" (UID: \"51fb28cb-f191-418e-ae54-43ca310712ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n" Jul 28 16:21:58.427322 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.427190 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:21:58.427322 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.427215 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqktx\" (UniqueName: \"kubernetes.io/projected/ce70be22-819d-4675-9c98-380ffd733f4c-kube-api-access-hqktx\") pod \"volume-data-source-validator-549c6d54cf-2m7fh\" (UID: \"ce70be22-819d-4675-9c98-380ffd733f4c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2m7fh" Jul 28 16:21:58.427322 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.427264 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/676a1e19-6f6c-42a4-881c-730a584c1c2b-tmp\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.427322 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.427308 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/676a1e19-6f6c-42a4-881c-730a584c1c2b-serving-cert\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.427459 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.427371 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/676a1e19-6f6c-42a4-881c-730a584c1c2b-service-ca-bundle\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.427459 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.427404 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/51fb28cb-f191-418e-ae54-43ca310712ec-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n\" (UID: \"51fb28cb-f191-418e-ae54-43ca310712ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n" Jul 28 16:21:58.427459 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.427437 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-stats-auth\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:21:58.427567 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.427473 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/54e904f0-e7ab-4882-9b0d-a91248f578a4-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-fz4bt\" (UID: \"54e904f0-e7ab-4882-9b0d-a91248f578a4\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:21:58.427567 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.427494 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/51fb28cb-f191-418e-ae54-43ca310712ec-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n\" (UID: \"51fb28cb-f191-418e-ae54-43ca310712ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n" Jul 28 16:21:58.427567 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.427519 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/676a1e19-6f6c-42a4-881c-730a584c1c2b-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.427674 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.427564 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/676a1e19-6f6c-42a4-881c-730a584c1c2b-snapshots\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.427866 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.427848 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/676a1e19-6f6c-42a4-881c-730a584c1c2b-service-ca-bundle\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.428379 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.428350 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/676a1e19-6f6c-42a4-881c-730a584c1c2b-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.431294 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.431266 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/676a1e19-6f6c-42a4-881c-730a584c1c2b-serving-cert\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.434806 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.434778 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqktx\" (UniqueName: \"kubernetes.io/projected/ce70be22-819d-4675-9c98-380ffd733f4c-kube-api-access-hqktx\") pod \"volume-data-source-validator-549c6d54cf-2m7fh\" (UID: \"ce70be22-819d-4675-9c98-380ffd733f4c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2m7fh" Jul 28 16:21:58.434890 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.434873 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8842\" (UniqueName: \"kubernetes.io/projected/676a1e19-6f6c-42a4-881c-730a584c1c2b-kube-api-access-g8842\") pod \"insights-operator-5885fdb759-9hwqx\" (UID: \"676a1e19-6f6c-42a4-881c-730a584c1c2b\") " pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.480322 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.480293 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2m7fh" Jul 28 16:21:58.488465 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.488440 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-9hwqx" Jul 28 16:21:58.528449 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528145 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9dl8\" (UniqueName: \"kubernetes.io/projected/ed0c86cb-26a6-4633-9170-9a042a3a71b3-kube-api-access-b9dl8\") pod \"console-operator-584dcb6b54-fcmgz\" (UID: \"ed0c86cb-26a6-4633-9170-9a042a3a71b3\") " pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:21:58.528449 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528198 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:21:58.528449 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528227 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fz4bt\" (UID: \"54e904f0-e7ab-4882-9b0d-a91248f578a4\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:21:58.528449 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528279 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-default-certificate\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:21:58.528449 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528306 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6bn7\" (UniqueName: \"kubernetes.io/projected/cab67026-e646-4a71-9c1a-e21d1fda8536-kube-api-access-g6bn7\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:21:58.528449 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528331 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2272\" (UniqueName: \"kubernetes.io/projected/5a015eff-1cd4-49e7-a328-0f54ea652ebf-kube-api-access-w2272\") pod \"cluster-samples-operator-6748489656-hjwt7\" (UID: \"5a015eff-1cd4-49e7-a328-0f54ea652ebf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" Jul 28 16:21:58.528449 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528363 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ed0c86cb-26a6-4633-9170-9a042a3a71b3-serving-cert\") pod \"console-operator-584dcb6b54-fcmgz\" (UID: \"ed0c86cb-26a6-4633-9170-9a042a3a71b3\") " pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:21:58.528449 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528389 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhblr\" (UniqueName: \"kubernetes.io/projected/54e904f0-e7ab-4882-9b0d-a91248f578a4-kube-api-access-jhblr\") pod \"cluster-monitoring-operator-548b489cb8-fz4bt\" (UID: \"54e904f0-e7ab-4882-9b0d-a91248f578a4\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:21:58.528449 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528418 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hjwt7\" (UID: \"5a015eff-1cd4-49e7-a328-0f54ea652ebf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" Jul 28 16:21:58.529260 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528470 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ed0c86cb-26a6-4633-9170-9a042a3a71b3-config\") pod \"console-operator-584dcb6b54-fcmgz\" (UID: \"ed0c86cb-26a6-4633-9170-9a042a3a71b3\") " pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:21:58.529260 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528499 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqwcs\" (UniqueName: \"kubernetes.io/projected/51fb28cb-f191-418e-ae54-43ca310712ec-kube-api-access-gqwcs\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n\" (UID: \"51fb28cb-f191-418e-ae54-43ca310712ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n" Jul 28 16:21:58.529260 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:58.528604 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 16:21:58.529260 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:58.528630 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 16:21:58.529260 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:58.528673 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls podName:54e904f0-e7ab-4882-9b0d-a91248f578a4 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:59.028657283 +0000 UTC m=+60.410846882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fz4bt" (UID: "54e904f0-e7ab-4882-9b0d-a91248f578a4") : secret "cluster-monitoring-operator-tls" not found Jul 28 16:21:58.529260 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528601 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:21:58.529260 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:58.528699 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs podName:cab67026-e646-4a71-9c1a-e21d1fda8536 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:59.028681942 +0000 UTC m=+60.410871551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs") pod "router-default-855657b557-78fr5" (UID: "cab67026-e646-4a71-9c1a-e21d1fda8536") : secret "router-metrics-certs-default" not found Jul 28 16:21:58.529260 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:58.528724 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle podName:cab67026-e646-4a71-9c1a-e21d1fda8536 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:59.028714695 +0000 UTC m=+60.410904488 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle") pod "router-default-855657b557-78fr5" (UID: "cab67026-e646-4a71-9c1a-e21d1fda8536") : configmap references non-existent config key: service-ca.crt Jul 28 16:21:58.529260 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528789 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed0c86cb-26a6-4633-9170-9a042a3a71b3-trusted-ca\") pod \"console-operator-584dcb6b54-fcmgz\" (UID: \"ed0c86cb-26a6-4633-9170-9a042a3a71b3\") " pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:21:58.529260 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528820 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/51fb28cb-f191-418e-ae54-43ca310712ec-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n\" (UID: \"51fb28cb-f191-418e-ae54-43ca310712ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n" Jul 28 16:21:58.529260 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528858 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-stats-auth\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:21:58.529260 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528876 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/54e904f0-e7ab-4882-9b0d-a91248f578a4-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-fz4bt\" (UID: \"54e904f0-e7ab-4882-9b0d-a91248f578a4\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:21:58.529260 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.528891 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/51fb28cb-f191-418e-ae54-43ca310712ec-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n\" (UID: \"51fb28cb-f191-418e-ae54-43ca310712ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n" Jul 28 16:21:58.529788 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.529410 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/51fb28cb-f191-418e-ae54-43ca310712ec-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n\" (UID: \"51fb28cb-f191-418e-ae54-43ca310712ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n" Jul 28 16:21:58.529843 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.529774 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/54e904f0-e7ab-4882-9b0d-a91248f578a4-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-fz4bt\" (UID: \"54e904f0-e7ab-4882-9b0d-a91248f578a4\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:21:58.531401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.531383 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/51fb28cb-f191-418e-ae54-43ca310712ec-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n\" (UID: \"51fb28cb-f191-418e-ae54-43ca310712ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n" Jul 28 16:21:58.532083 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.532061 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-stats-auth\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:21:58.532179 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.532083 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-default-certificate\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:21:58.537594 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.537567 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqwcs\" (UniqueName: \"kubernetes.io/projected/51fb28cb-f191-418e-ae54-43ca310712ec-kube-api-access-gqwcs\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n\" (UID: \"51fb28cb-f191-418e-ae54-43ca310712ec\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n" Jul 28 16:21:58.537716 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.537630 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhblr\" (UniqueName: \"kubernetes.io/projected/54e904f0-e7ab-4882-9b0d-a91248f578a4-kube-api-access-jhblr\") pod \"cluster-monitoring-operator-548b489cb8-fz4bt\" (UID: \"54e904f0-e7ab-4882-9b0d-a91248f578a4\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:21:58.539244 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.539216 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6bn7\" (UniqueName: \"kubernetes.io/projected/cab67026-e646-4a71-9c1a-e21d1fda8536-kube-api-access-g6bn7\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:21:58.586174 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.586145 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n" Jul 28 16:21:58.626776 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.626702 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2m7fh"] Jul 28 16:21:58.629518 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.629493 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ed0c86cb-26a6-4633-9170-9a042a3a71b3-config\") pod \"console-operator-584dcb6b54-fcmgz\" (UID: \"ed0c86cb-26a6-4633-9170-9a042a3a71b3\") " pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:21:58.629645 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.629608 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed0c86cb-26a6-4633-9170-9a042a3a71b3-trusted-ca\") pod \"console-operator-584dcb6b54-fcmgz\" (UID: \"ed0c86cb-26a6-4633-9170-9a042a3a71b3\") " pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:21:58.629708 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.629657 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9dl8\" (UniqueName: \"kubernetes.io/projected/ed0c86cb-26a6-4633-9170-9a042a3a71b3-kube-api-access-b9dl8\") pod \"console-operator-584dcb6b54-fcmgz\" (UID: \"ed0c86cb-26a6-4633-9170-9a042a3a71b3\") " pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:21:58.629877 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.629856 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2272\" (UniqueName: \"kubernetes.io/projected/5a015eff-1cd4-49e7-a328-0f54ea652ebf-kube-api-access-w2272\") pod \"cluster-samples-operator-6748489656-hjwt7\" (UID: \"5a015eff-1cd4-49e7-a328-0f54ea652ebf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" Jul 28 16:21:58.630009 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.629900 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ed0c86cb-26a6-4633-9170-9a042a3a71b3-serving-cert\") pod \"console-operator-584dcb6b54-fcmgz\" (UID: \"ed0c86cb-26a6-4633-9170-9a042a3a71b3\") " pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:21:58.630009 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.629922 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hjwt7\" (UID: \"5a015eff-1cd4-49e7-a328-0f54ea652ebf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" Jul 28 16:21:58.630116 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:58.630035 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 16:21:58.630116 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:58.630082 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls podName:5a015eff-1cd4-49e7-a328-0f54ea652ebf nodeName:}" failed. No retries permitted until 2026-07-28 16:21:59.130070307 +0000 UTC m=+60.512259903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls") pod "cluster-samples-operator-6748489656-hjwt7" (UID: "5a015eff-1cd4-49e7-a328-0f54ea652ebf") : secret "samples-operator-tls" not found Jul 28 16:21:58.630305 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.630283 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ed0c86cb-26a6-4633-9170-9a042a3a71b3-config\") pod \"console-operator-584dcb6b54-fcmgz\" (UID: \"ed0c86cb-26a6-4633-9170-9a042a3a71b3\") " pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:21:58.630730 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.630703 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed0c86cb-26a6-4633-9170-9a042a3a71b3-trusted-ca\") pod \"console-operator-584dcb6b54-fcmgz\" (UID: \"ed0c86cb-26a6-4633-9170-9a042a3a71b3\") " pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:21:58.631002 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:21:58.630960 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce70be22_819d_4675_9c98_380ffd733f4c.slice/crio-1dd6be453c417d293def16ae8d0fd18dc6e4d0bb9b96fb2d2f5d80a2d7111b91 WatchSource:0}: Error finding container 1dd6be453c417d293def16ae8d0fd18dc6e4d0bb9b96fb2d2f5d80a2d7111b91: Status 404 returned error can't find the container with id 1dd6be453c417d293def16ae8d0fd18dc6e4d0bb9b96fb2d2f5d80a2d7111b91 Jul 28 16:21:58.632640 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.632608 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ed0c86cb-26a6-4633-9170-9a042a3a71b3-serving-cert\") pod \"console-operator-584dcb6b54-fcmgz\" (UID: \"ed0c86cb-26a6-4633-9170-9a042a3a71b3\") " pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:21:58.641278 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.641248 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2272\" (UniqueName: \"kubernetes.io/projected/5a015eff-1cd4-49e7-a328-0f54ea652ebf-kube-api-access-w2272\") pod \"cluster-samples-operator-6748489656-hjwt7\" (UID: \"5a015eff-1cd4-49e7-a328-0f54ea652ebf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" Jul 28 16:21:58.642076 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.642059 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9dl8\" (UniqueName: \"kubernetes.io/projected/ed0c86cb-26a6-4633-9170-9a042a3a71b3-kube-api-access-b9dl8\") pod \"console-operator-584dcb6b54-fcmgz\" (UID: \"ed0c86cb-26a6-4633-9170-9a042a3a71b3\") " pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:21:58.645709 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.645687 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-9hwqx"] Jul 28 16:21:58.649454 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:21:58.649428 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod676a1e19_6f6c_42a4_881c_730a584c1c2b.slice/crio-5d0052795f8bd8a2e7a74b17bbcdc5a4ef212a96619191ebafefb3bf4485f9e5 WatchSource:0}: Error finding container 5d0052795f8bd8a2e7a74b17bbcdc5a4ef212a96619191ebafefb3bf4485f9e5: Status 404 returned error can't find the container with id 5d0052795f8bd8a2e7a74b17bbcdc5a4ef212a96619191ebafefb3bf4485f9e5 Jul 28 16:21:58.690191 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.690108 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:21:58.743386 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.743225 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n"] Jul 28 16:21:58.746488 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:21:58.746463 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51fb28cb_f191_418e_ae54_43ca310712ec.slice/crio-a49e2a9102f6b5698ca3f5d49e95d127c1b3d9b21c818e56ac101b37f9c8d091 WatchSource:0}: Error finding container a49e2a9102f6b5698ca3f5d49e95d127c1b3d9b21c818e56ac101b37f9c8d091: Status 404 returned error can't find the container with id a49e2a9102f6b5698ca3f5d49e95d127c1b3d9b21c818e56ac101b37f9c8d091 Jul 28 16:21:58.825847 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:58.825752 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-fcmgz"] Jul 28 16:21:58.828181 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:21:58.828156 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded0c86cb_26a6_4633_9170_9a042a3a71b3.slice/crio-c301fa8ca79a142f10e1e5cbc18b74587d399511e6dec93864ab5847c984f9a1 WatchSource:0}: Error finding container c301fa8ca79a142f10e1e5cbc18b74587d399511e6dec93864ab5847c984f9a1: Status 404 returned error can't find the container with id c301fa8ca79a142f10e1e5cbc18b74587d399511e6dec93864ab5847c984f9a1 Jul 28 16:21:59.033327 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:59.033299 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:21:59.033506 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:59.033339 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fz4bt\" (UID: \"54e904f0-e7ab-4882-9b0d-a91248f578a4\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:21:59.033506 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:59.033451 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 16:21:59.033506 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:59.033461 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:21:59.033506 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:59.033476 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 16:21:59.033706 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:59.033515 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs podName:cab67026-e646-4a71-9c1a-e21d1fda8536 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:00.033496762 +0000 UTC m=+61.415686360 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs") pod "router-default-855657b557-78fr5" (UID: "cab67026-e646-4a71-9c1a-e21d1fda8536") : secret "router-metrics-certs-default" not found Jul 28 16:21:59.033706 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:59.033573 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls podName:54e904f0-e7ab-4882-9b0d-a91248f578a4 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:00.033563907 +0000 UTC m=+61.415753504 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fz4bt" (UID: "54e904f0-e7ab-4882-9b0d-a91248f578a4") : secret "cluster-monitoring-operator-tls" not found Jul 28 16:21:59.033706 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:59.033588 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle podName:cab67026-e646-4a71-9c1a-e21d1fda8536 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:00.03358039 +0000 UTC m=+61.415769986 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle") pod "router-default-855657b557-78fr5" (UID: "cab67026-e646-4a71-9c1a-e21d1fda8536") : configmap references non-existent config key: service-ca.crt Jul 28 16:21:59.133931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:59.133899 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hjwt7\" (UID: \"5a015eff-1cd4-49e7-a328-0f54ea652ebf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" Jul 28 16:21:59.134132 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:59.134092 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 16:21:59.134260 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:21:59.134160 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls podName:5a015eff-1cd4-49e7-a328-0f54ea652ebf nodeName:}" failed. No retries permitted until 2026-07-28 16:22:00.134140212 +0000 UTC m=+61.516329811 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls") pod "cluster-samples-operator-6748489656-hjwt7" (UID: "5a015eff-1cd4-49e7-a328-0f54ea652ebf") : secret "samples-operator-tls" not found Jul 28 16:21:59.469602 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:59.469446 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n" event={"ID":"51fb28cb-f191-418e-ae54-43ca310712ec","Type":"ContainerStarted","Data":"a49e2a9102f6b5698ca3f5d49e95d127c1b3d9b21c818e56ac101b37f9c8d091"} Jul 28 16:21:59.472396 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:59.472285 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-9hwqx" event={"ID":"676a1e19-6f6c-42a4-881c-730a584c1c2b","Type":"ContainerStarted","Data":"5d0052795f8bd8a2e7a74b17bbcdc5a4ef212a96619191ebafefb3bf4485f9e5"} Jul 28 16:21:59.474629 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:59.474588 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2m7fh" event={"ID":"ce70be22-819d-4675-9c98-380ffd733f4c","Type":"ContainerStarted","Data":"1dd6be453c417d293def16ae8d0fd18dc6e4d0bb9b96fb2d2f5d80a2d7111b91"} Jul 28 16:21:59.476209 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:21:59.476151 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" event={"ID":"ed0c86cb-26a6-4633-9170-9a042a3a71b3","Type":"ContainerStarted","Data":"c301fa8ca79a142f10e1e5cbc18b74587d399511e6dec93864ab5847c984f9a1"} Jul 28 16:22:00.042931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:00.042882 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:22:00.043116 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:00.042945 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fz4bt\" (UID: \"54e904f0-e7ab-4882-9b0d-a91248f578a4\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:22:00.043116 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:00.043080 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:22:00.043238 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:00.043222 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle podName:cab67026-e646-4a71-9c1a-e21d1fda8536 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:02.043205816 +0000 UTC m=+63.425395412 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle") pod "router-default-855657b557-78fr5" (UID: "cab67026-e646-4a71-9c1a-e21d1fda8536") : configmap references non-existent config key: service-ca.crt Jul 28 16:22:00.043613 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:00.043569 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 16:22:00.043684 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:00.043622 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs podName:cab67026-e646-4a71-9c1a-e21d1fda8536 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:02.043607119 +0000 UTC m=+63.425796732 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs") pod "router-default-855657b557-78fr5" (UID: "cab67026-e646-4a71-9c1a-e21d1fda8536") : secret "router-metrics-certs-default" not found Jul 28 16:22:00.043741 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:00.043682 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 16:22:00.043741 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:00.043718 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls podName:54e904f0-e7ab-4882-9b0d-a91248f578a4 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:02.043707255 +0000 UTC m=+63.425896867 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fz4bt" (UID: "54e904f0-e7ab-4882-9b0d-a91248f578a4") : secret "cluster-monitoring-operator-tls" not found Jul 28 16:22:00.144408 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:00.144321 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hjwt7\" (UID: \"5a015eff-1cd4-49e7-a328-0f54ea652ebf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" Jul 28 16:22:00.144578 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:00.144492 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 16:22:00.144578 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:00.144569 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls podName:5a015eff-1cd4-49e7-a328-0f54ea652ebf nodeName:}" failed. No retries permitted until 2026-07-28 16:22:02.144547477 +0000 UTC m=+63.526737079 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls") pod "cluster-samples-operator-6748489656-hjwt7" (UID: "5a015eff-1cd4-49e7-a328-0f54ea652ebf") : secret "samples-operator-tls" not found Jul 28 16:22:00.396417 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:00.396338 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xx7xl" Jul 28 16:22:00.480991 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:00.480938 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2m7fh" event={"ID":"ce70be22-819d-4675-9c98-380ffd733f4c","Type":"ContainerStarted","Data":"5c89ef2057afb2518ebf966547a53128abc21e296662113f70420425491e9161"} Jul 28 16:22:00.500369 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:00.500298 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-2m7fh" podStartSLOduration=1.2697052979999999 podStartE2EDuration="2.500275347s" totalImagesPullingTime="1.230570049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:00.498797495 +0000 UTC m=+61.880987137" watchObservedRunningTime="2026-07-28 16:22:00.500275347 +0000 UTC m=+61.882464968" Jul 28 16:22:02.058396 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.058361 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:22:02.058738 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.058405 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fz4bt\" (UID: \"54e904f0-e7ab-4882-9b0d-a91248f578a4\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:22:02.058738 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.058482 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:22:02.058738 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:02.058503 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 16:22:02.058738 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:02.058542 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 16:22:02.058738 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:02.058579 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs podName:cab67026-e646-4a71-9c1a-e21d1fda8536 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:06.058556992 +0000 UTC m=+67.440746601 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs") pod "router-default-855657b557-78fr5" (UID: "cab67026-e646-4a71-9c1a-e21d1fda8536") : secret "router-metrics-certs-default" not found Jul 28 16:22:02.058738 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:02.058595 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls podName:54e904f0-e7ab-4882-9b0d-a91248f578a4 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:06.058588182 +0000 UTC m=+67.440777778 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fz4bt" (UID: "54e904f0-e7ab-4882-9b0d-a91248f578a4") : secret "cluster-monitoring-operator-tls" not found Jul 28 16:22:02.058738 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:02.058605 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle podName:cab67026-e646-4a71-9c1a-e21d1fda8536 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:06.058599699 +0000 UTC m=+67.440789295 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle") pod "router-default-855657b557-78fr5" (UID: "cab67026-e646-4a71-9c1a-e21d1fda8536") : configmap references non-existent config key: service-ca.crt Jul 28 16:22:02.159709 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.159621 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hjwt7\" (UID: \"5a015eff-1cd4-49e7-a328-0f54ea652ebf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" Jul 28 16:22:02.159861 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:02.159805 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 16:22:02.159915 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:02.159865 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls podName:5a015eff-1cd4-49e7-a328-0f54ea652ebf nodeName:}" failed. No retries permitted until 2026-07-28 16:22:06.15984816 +0000 UTC m=+67.542037759 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls") pod "cluster-samples-operator-6748489656-hjwt7" (UID: "5a015eff-1cd4-49e7-a328-0f54ea652ebf") : secret "samples-operator-tls" not found Jul 28 16:22:02.489739 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.489706 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/0.log" Jul 28 16:22:02.490025 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.489758 2566 generic.go:356] "Generic (PLEG): container finished" podID="ed0c86cb-26a6-4633-9170-9a042a3a71b3" containerID="19be90ea579fd57fde664a41d68c6890c49bd47bf553a6f2df952560f860d298" exitCode=255 Jul 28 16:22:02.490197 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.490184 2566 scope.go:117] "RemoveContainer" containerID="19be90ea579fd57fde664a41d68c6890c49bd47bf553a6f2df952560f860d298" Jul 28 16:22:02.490469 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.490445 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" event={"ID":"ed0c86cb-26a6-4633-9170-9a042a3a71b3","Type":"ContainerDied","Data":"19be90ea579fd57fde664a41d68c6890c49bd47bf553a6f2df952560f860d298"} Jul 28 16:22:02.497335 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.495004 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n" event={"ID":"51fb28cb-f191-418e-ae54-43ca310712ec","Type":"ContainerStarted","Data":"df392f63b90bc2b22cf8d4b1b5f33b8c0cec752f0f8f66a337f98f6e467d373b"} Jul 28 16:22:02.497725 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.497656 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-9hwqx" event={"ID":"676a1e19-6f6c-42a4-881c-730a584c1c2b","Type":"ContainerStarted","Data":"2d016f4bc80dc2d28ca87b90b6402b08276da08b01ecdbf6bb6162a81f657c7d"} Jul 28 16:22:02.511175 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.511141 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-chq49"] Jul 28 16:22:02.521897 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.521026 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:02.523801 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.523775 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lx2xj\"" Jul 28 16:22:02.526033 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.525991 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 28 16:22:02.526033 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.525998 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 28 16:22:02.540370 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.540343 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-chq49"] Jul 28 16:22:02.548068 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.547637 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-9hwqx" podStartSLOduration=1.295013358 podStartE2EDuration="4.547615491s" totalImagesPullingTime="3.252602133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:02.546848196 +0000 UTC m=+63.929037814" watchObservedRunningTime="2026-07-28 16:22:02.547615491 +0000 UTC m=+63.929805111" Jul 28 16:22:02.563626 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.563594 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:02.563782 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.563670 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/63243f80-5a2a-4991-91e4-3f217aa9c2cb-crio-socket\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:02.564457 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.564430 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7574p\" (UniqueName: \"kubernetes.io/projected/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-api-access-7574p\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:02.565073 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.564598 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/63243f80-5a2a-4991-91e4-3f217aa9c2cb-data-volume\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:02.565073 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.564709 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:02.573018 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.572959 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n" podStartSLOduration=1.422513725 podStartE2EDuration="4.572941908s" totalImagesPullingTime="3.150428183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:02.572008887 +0000 UTC m=+63.954198502" watchObservedRunningTime="2026-07-28 16:22:02.572941908 +0000 UTC m=+63.955131525" Jul 28 16:22:02.665769 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.665729 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:02.665964 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.665820 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:02.665964 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.665859 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/63243f80-5a2a-4991-91e4-3f217aa9c2cb-crio-socket\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:02.665964 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:02.665873 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 16:22:02.665964 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.665925 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7574p\" (UniqueName: \"kubernetes.io/projected/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-api-access-7574p\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:02.665964 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:02.665957 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm podName:63243f80-5a2a-4991-91e4-3f217aa9c2cb nodeName:}" failed. No retries permitted until 2026-07-28 16:22:03.165934418 +0000 UTC m=+64.548124028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-chq49" (UID: "63243f80-5a2a-4991-91e4-3f217aa9c2cb") : configmap "kube-rbac-proxy" not found Jul 28 16:22:02.666232 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:02.666027 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 16:22:02.666232 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.666040 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/63243f80-5a2a-4991-91e4-3f217aa9c2cb-crio-socket\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:02.666232 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:02.666086 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls podName:63243f80-5a2a-4991-91e4-3f217aa9c2cb nodeName:}" failed. No retries permitted until 2026-07-28 16:22:03.166069818 +0000 UTC m=+64.548259414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-chq49" (UID: "63243f80-5a2a-4991-91e4-3f217aa9c2cb") : secret "insights-runtime-extractor-tls" not found Jul 28 16:22:02.666232 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.666104 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/63243f80-5a2a-4991-91e4-3f217aa9c2cb-data-volume\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:02.666445 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.666427 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/63243f80-5a2a-4991-91e4-3f217aa9c2cb-data-volume\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:02.674283 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:02.674260 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7574p\" (UniqueName: \"kubernetes.io/projected/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-api-access-7574p\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:03.170721 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:03.170675 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:03.171165 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:03.170752 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:03.171165 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:03.170805 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 16:22:03.171165 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:03.170877 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm podName:63243f80-5a2a-4991-91e4-3f217aa9c2cb nodeName:}" failed. No retries permitted until 2026-07-28 16:22:04.170858478 +0000 UTC m=+65.553048094 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-chq49" (UID: "63243f80-5a2a-4991-91e4-3f217aa9c2cb") : configmap "kube-rbac-proxy" not found Jul 28 16:22:03.171165 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:03.170879 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 16:22:03.171165 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:03.170914 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls podName:63243f80-5a2a-4991-91e4-3f217aa9c2cb nodeName:}" failed. No retries permitted until 2026-07-28 16:22:04.170904009 +0000 UTC m=+65.553093619 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-chq49" (UID: "63243f80-5a2a-4991-91e4-3f217aa9c2cb") : secret "insights-runtime-extractor-tls" not found Jul 28 16:22:03.502656 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:03.502625 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/1.log" Jul 28 16:22:03.503133 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:03.503119 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/0.log" Jul 28 16:22:03.503201 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:03.503154 2566 generic.go:356] "Generic (PLEG): container finished" podID="ed0c86cb-26a6-4633-9170-9a042a3a71b3" containerID="dbb3a887a80e30387f9972360f614e13b2cdc1a7b839f5c2168cbf91b2c27ab0" exitCode=255 Jul 28 16:22:03.503257 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:03.503239 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" event={"ID":"ed0c86cb-26a6-4633-9170-9a042a3a71b3","Type":"ContainerDied","Data":"dbb3a887a80e30387f9972360f614e13b2cdc1a7b839f5c2168cbf91b2c27ab0"} Jul 28 16:22:03.504171 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:03.504152 2566 scope.go:117] "RemoveContainer" containerID="19be90ea579fd57fde664a41d68c6890c49bd47bf553a6f2df952560f860d298" Jul 28 16:22:03.504275 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:03.504248 2566 scope.go:117] "RemoveContainer" containerID="dbb3a887a80e30387f9972360f614e13b2cdc1a7b839f5c2168cbf91b2c27ab0" Jul 28 16:22:03.504411 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:03.504387 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-fcmgz_openshift-console-operator(ed0c86cb-26a6-4633-9170-9a042a3a71b3)\"" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" podUID="ed0c86cb-26a6-4633-9170-9a042a3a71b3" Jul 28 16:22:04.181400 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.181368 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:04.181854 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.181437 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:04.181854 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:04.181487 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 16:22:04.181854 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:04.181552 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm podName:63243f80-5a2a-4991-91e4-3f217aa9c2cb nodeName:}" failed. No retries permitted until 2026-07-28 16:22:06.181533843 +0000 UTC m=+67.563723459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-chq49" (UID: "63243f80-5a2a-4991-91e4-3f217aa9c2cb") : configmap "kube-rbac-proxy" not found Jul 28 16:22:04.181854 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:04.181625 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 16:22:04.181854 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:04.181705 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls podName:63243f80-5a2a-4991-91e4-3f217aa9c2cb nodeName:}" failed. No retries permitted until 2026-07-28 16:22:06.181685987 +0000 UTC m=+67.563875599 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-chq49" (UID: "63243f80-5a2a-4991-91e4-3f217aa9c2cb") : secret "insights-runtime-extractor-tls" not found Jul 28 16:22:04.508023 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.507994 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/1.log" Jul 28 16:22:04.508336 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.508321 2566 scope.go:117] "RemoveContainer" containerID="dbb3a887a80e30387f9972360f614e13b2cdc1a7b839f5c2168cbf91b2c27ab0" Jul 28 16:22:04.508489 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:04.508472 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-fcmgz_openshift-console-operator(ed0c86cb-26a6-4633-9170-9a042a3a71b3)\"" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" podUID="ed0c86cb-26a6-4633-9170-9a042a3a71b3" Jul 28 16:22:04.515771 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.515742 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8fpkf_4d2caa30-ece8-42cf-9dc4-e45e56dae4a2/dns-node-resolver/0.log" Jul 28 16:22:04.787700 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.787614 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64ljk\" (UniqueName: \"kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk\") pod \"network-check-target-gbdb4\" (UID: \"6d6d1243-b8d6-4402-96f1-5350263231ab\") " pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:22:04.787700 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.787675 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs\") pod \"network-metrics-daemon-gb4cw\" (UID: \"4a8f2962-0386-4066-a462-445cbb6c3f09\") " pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:22:04.787937 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.787725 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret\") pod \"global-pull-secret-syncer-nzcls\" (UID: \"02ce611d-aef7-4996-bb0f-80279da441b5\") " pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:22:04.790259 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.790238 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 16:22:04.790381 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.790238 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 16:22:04.790381 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.790316 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 16:22:04.798301 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:04.798282 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 28 16:22:04.798403 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:04.798338 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs podName:4a8f2962-0386-4066-a462-445cbb6c3f09 nodeName:}" failed. No retries permitted until 2026-07-28 16:23:08.798320948 +0000 UTC m=+130.180510543 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs") pod "network-metrics-daemon-gb4cw" (UID: "4a8f2962-0386-4066-a462-445cbb6c3f09") : secret "metrics-daemon-secret" not found Jul 28 16:22:04.800272 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.800250 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 16:22:04.800651 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.800630 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/02ce611d-aef7-4996-bb0f-80279da441b5-original-pull-secret\") pod \"global-pull-secret-syncer-nzcls\" (UID: \"02ce611d-aef7-4996-bb0f-80279da441b5\") " pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:22:04.811883 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.811855 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64ljk\" (UniqueName: \"kubernetes.io/projected/6d6d1243-b8d6-4402-96f1-5350263231ab-kube-api-access-64ljk\") pod \"network-check-target-gbdb4\" (UID: \"6d6d1243-b8d6-4402-96f1-5350263231ab\") " pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:22:04.917088 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.917057 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gnm96\"" Jul 28 16:22:04.922142 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.922116 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzcls" Jul 28 16:22:04.925243 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:04.925219 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:22:05.089699 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:05.089664 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nzcls"] Jul 28 16:22:05.092807 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:22:05.092771 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02ce611d_aef7_4996_bb0f_80279da441b5.slice/crio-97ad0a36b9b9ca840584588daa08214399a967ffe5a4cdd150106b0256a891e6 WatchSource:0}: Error finding container 97ad0a36b9b9ca840584588daa08214399a967ffe5a4cdd150106b0256a891e6: Status 404 returned error can't find the container with id 97ad0a36b9b9ca840584588daa08214399a967ffe5a4cdd150106b0256a891e6 Jul 28 16:22:05.107535 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:05.107511 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gbdb4"] Jul 28 16:22:05.110083 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:22:05.110058 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d6d1243_b8d6_4402_96f1_5350263231ab.slice/crio-325d9bc52141555a3b3de00ce10b490bd5a3e74c74e9cc287a7d479321a5f5de WatchSource:0}: Error finding container 325d9bc52141555a3b3de00ce10b490bd5a3e74c74e9cc287a7d479321a5f5de: Status 404 returned error can't find the container with id 325d9bc52141555a3b3de00ce10b490bd5a3e74c74e9cc287a7d479321a5f5de Jul 28 16:22:05.314526 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:05.312524 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6jp82_18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b/node-ca/0.log" Jul 28 16:22:05.512447 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:05.512408 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gbdb4" event={"ID":"6d6d1243-b8d6-4402-96f1-5350263231ab","Type":"ContainerStarted","Data":"325d9bc52141555a3b3de00ce10b490bd5a3e74c74e9cc287a7d479321a5f5de"} Jul 28 16:22:05.513367 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:05.513342 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nzcls" event={"ID":"02ce611d-aef7-4996-bb0f-80279da441b5","Type":"ContainerStarted","Data":"97ad0a36b9b9ca840584588daa08214399a967ffe5a4cdd150106b0256a891e6"} Jul 28 16:22:06.100014 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:06.099950 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:22:06.100184 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:06.100029 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fz4bt\" (UID: \"54e904f0-e7ab-4882-9b0d-a91248f578a4\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:22:06.100184 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:06.100158 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:22:06.100342 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:06.100327 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle podName:cab67026-e646-4a71-9c1a-e21d1fda8536 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:14.100307889 +0000 UTC m=+75.482497506 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle") pod "router-default-855657b557-78fr5" (UID: "cab67026-e646-4a71-9c1a-e21d1fda8536") : configmap references non-existent config key: service-ca.crt Jul 28 16:22:06.100769 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:06.100742 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 28 16:22:06.100843 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:06.100791 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs podName:cab67026-e646-4a71-9c1a-e21d1fda8536 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:14.100776333 +0000 UTC m=+75.482965942 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs") pod "router-default-855657b557-78fr5" (UID: "cab67026-e646-4a71-9c1a-e21d1fda8536") : secret "router-metrics-certs-default" not found Jul 28 16:22:06.100843 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:06.100836 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 16:22:06.100948 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:06.100861 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls podName:54e904f0-e7ab-4882-9b0d-a91248f578a4 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:14.100851161 +0000 UTC m=+75.483040756 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fz4bt" (UID: "54e904f0-e7ab-4882-9b0d-a91248f578a4") : secret "cluster-monitoring-operator-tls" not found Jul 28 16:22:06.201379 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:06.201343 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hjwt7\" (UID: \"5a015eff-1cd4-49e7-a328-0f54ea652ebf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" Jul 28 16:22:06.201580 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:06.201396 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:06.201580 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:06.201457 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:06.201580 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:06.201565 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 16:22:06.201739 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:06.201629 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls podName:63243f80-5a2a-4991-91e4-3f217aa9c2cb nodeName:}" failed. No retries permitted until 2026-07-28 16:22:10.20161076 +0000 UTC m=+71.583800360 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-chq49" (UID: "63243f80-5a2a-4991-91e4-3f217aa9c2cb") : secret "insights-runtime-extractor-tls" not found Jul 28 16:22:06.201988 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:06.201949 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 16:22:06.202133 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:06.202006 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls podName:5a015eff-1cd4-49e7-a328-0f54ea652ebf nodeName:}" failed. No retries permitted until 2026-07-28 16:22:14.201992229 +0000 UTC m=+75.584181825 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls") pod "cluster-samples-operator-6748489656-hjwt7" (UID: "5a015eff-1cd4-49e7-a328-0f54ea652ebf") : secret "samples-operator-tls" not found Jul 28 16:22:06.202133 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:06.202050 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 16:22:06.202133 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:06.202083 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm podName:63243f80-5a2a-4991-91e4-3f217aa9c2cb nodeName:}" failed. No retries permitted until 2026-07-28 16:22:10.2020727 +0000 UTC m=+71.584262311 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-chq49" (UID: "63243f80-5a2a-4991-91e4-3f217aa9c2cb") : configmap "kube-rbac-proxy" not found Jul 28 16:22:06.403170 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:06.403081 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:22:06.403608 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:06.403183 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:22:06.403608 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:06.403353 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 16:22:06.403608 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:06.403413 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls podName:08c6af7a-3f2a-491b-aee3-61a1dafe2c0a nodeName:}" failed. No retries permitted until 2026-07-28 16:22:38.403394985 +0000 UTC m=+99.785584582 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls") pod "dns-default-fdhkh" (UID: "08c6af7a-3f2a-491b-aee3-61a1dafe2c0a") : secret "dns-default-metrics-tls" not found Jul 28 16:22:06.403842 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:06.403828 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 16:22:06.403896 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:06.403845 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-554f97cd94-4klh7: secret "image-registry-tls" not found Jul 28 16:22:06.403896 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:06.403894 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls podName:bae9c53c-7ff2-43b0-b0ac-5433a37338e4 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:38.403876925 +0000 UTC m=+99.786066541 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls") pod "image-registry-554f97cd94-4klh7" (UID: "bae9c53c-7ff2-43b0-b0ac-5433a37338e4") : secret "image-registry-tls" not found Jul 28 16:22:06.513617 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:06.513587 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tftts_7a43fcfe-07e8-4f38-bfff-fb4cdedb6d3b/serve-healthcheck-canary/0.log" Jul 28 16:22:08.690640 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:08.690601 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:22:08.691115 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:08.690691 2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:22:08.691115 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:08.691066 2566 scope.go:117] "RemoveContainer" containerID="dbb3a887a80e30387f9972360f614e13b2cdc1a7b839f5c2168cbf91b2c27ab0" Jul 28 16:22:08.691270 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:08.691250 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-fcmgz_openshift-console-operator(ed0c86cb-26a6-4633-9170-9a042a3a71b3)\"" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" podUID="ed0c86cb-26a6-4633-9170-9a042a3a71b3" Jul 28 16:22:09.526712 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:09.526686 2566 scope.go:117] "RemoveContainer" containerID="dbb3a887a80e30387f9972360f614e13b2cdc1a7b839f5c2168cbf91b2c27ab0" Jul 28 16:22:09.526880 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:09.526841 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-fcmgz_openshift-console-operator(ed0c86cb-26a6-4633-9170-9a042a3a71b3)\"" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" podUID="ed0c86cb-26a6-4633-9170-9a042a3a71b3" Jul 28 16:22:10.242823 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:10.242768 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:10.243252 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:10.242891 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:10.243252 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:10.242927 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 16:22:10.243252 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:10.242952 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 16:22:10.243252 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:10.243010 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls podName:63243f80-5a2a-4991-91e4-3f217aa9c2cb nodeName:}" failed. No retries permitted until 2026-07-28 16:22:18.242994127 +0000 UTC m=+79.625183727 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-chq49" (UID: "63243f80-5a2a-4991-91e4-3f217aa9c2cb") : secret "insights-runtime-extractor-tls" not found Jul 28 16:22:10.243252 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:10.243026 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm podName:63243f80-5a2a-4991-91e4-3f217aa9c2cb nodeName:}" failed. No retries permitted until 2026-07-28 16:22:18.243019219 +0000 UTC m=+79.625208818 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm") pod "insights-runtime-extractor-chq49" (UID: "63243f80-5a2a-4991-91e4-3f217aa9c2cb") : configmap "kube-rbac-proxy" not found Jul 28 16:22:10.531520 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:10.531413 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nzcls" event={"ID":"02ce611d-aef7-4996-bb0f-80279da441b5","Type":"ContainerStarted","Data":"90096a334c718278d78aacd0f11828e3ff770196faafb7accc13be4657416082"} Jul 28 16:22:10.532895 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:10.532866 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gbdb4" event={"ID":"6d6d1243-b8d6-4402-96f1-5350263231ab","Type":"ContainerStarted","Data":"7a463549892c60fb282f71c09d7e3d412193f6840fc7d612274101c41be57d40"} Jul 28 16:22:10.533030 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:10.532998 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:22:10.547611 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:10.547562 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nzcls" podStartSLOduration=66.096226113 podStartE2EDuration="1m10.547547242s" totalImagesPullingTime="4.451321129s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:10.545489352 +0000 UTC m=+71.927678994" watchObservedRunningTime="2026-07-28 16:22:10.547547242 +0000 UTC m=+71.929736868" Jul 28 16:22:10.560386 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:10.560340 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gbdb4" podStartSLOduration=66.130485743 podStartE2EDuration="1m10.560323975s" totalImagesPullingTime="4.429838232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:10.559181925 +0000 UTC m=+71.941371555" watchObservedRunningTime="2026-07-28 16:22:10.560323975 +0000 UTC m=+71.942513592" Jul 28 16:22:14.177652 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:14.177603 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:22:14.177652 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:14.177664 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fz4bt\" (UID: \"54e904f0-e7ab-4882-9b0d-a91248f578a4\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:22:14.178247 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:14.177801 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:22:14.178247 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:14.177901 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 16:22:14.178247 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:14.177957 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle podName:cab67026-e646-4a71-9c1a-e21d1fda8536 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:30.177941242 +0000 UTC m=+91.560130842 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle") pod "router-default-855657b557-78fr5" (UID: "cab67026-e646-4a71-9c1a-e21d1fda8536") : configmap references non-existent config key: service-ca.crt Jul 28 16:22:14.178247 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:14.177997 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls podName:54e904f0-e7ab-4882-9b0d-a91248f578a4 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:30.177990623 +0000 UTC m=+91.560180218 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-fz4bt" (UID: "54e904f0-e7ab-4882-9b0d-a91248f578a4") : secret "cluster-monitoring-operator-tls" not found Jul 28 16:22:14.180270 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:14.180254 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cab67026-e646-4a71-9c1a-e21d1fda8536-metrics-certs\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:22:14.278686 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:14.278637 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hjwt7\" (UID: \"5a015eff-1cd4-49e7-a328-0f54ea652ebf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" Jul 28 16:22:14.281276 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:14.281250 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5a015eff-1cd4-49e7-a328-0f54ea652ebf-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hjwt7\" (UID: \"5a015eff-1cd4-49e7-a328-0f54ea652ebf\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" Jul 28 16:22:14.552471 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:14.552443 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-f6p2c\"" Jul 28 16:22:14.560598 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:14.560579 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" Jul 28 16:22:14.692401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:14.692378 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7"] Jul 28 16:22:15.547917 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:15.547876 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" event={"ID":"5a015eff-1cd4-49e7-a328-0f54ea652ebf","Type":"ContainerStarted","Data":"db78a2c832370fb3c1f40b829ccb932898a56d98a7041fa69ba253d56250646e"} Jul 28 16:22:16.552676 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:16.552589 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" event={"ID":"5a015eff-1cd4-49e7-a328-0f54ea652ebf","Type":"ContainerStarted","Data":"60e94e552813ccab2cdd03d9d9e09feb4850284e61a400d08ea7f9adf821489d"} Jul 28 16:22:16.552676 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:16.552654 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" event={"ID":"5a015eff-1cd4-49e7-a328-0f54ea652ebf","Type":"ContainerStarted","Data":"98477844a8665696074d1316ea01cce27329daae2f56c3288e3685a90e66c520"} Jul 28 16:22:16.570724 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:16.570673 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hjwt7" podStartSLOduration=17.063631451 podStartE2EDuration="18.57065854s" totalImagesPullingTime="1.507027089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:16.569431738 +0000 UTC m=+77.951621356" watchObservedRunningTime="2026-07-28 16:22:16.57065854 +0000 UTC m=+77.952848158" Jul 28 16:22:18.312094 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:18.312052 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:18.312566 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:18.312118 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:18.312566 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:18.312318 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 16:22:18.312566 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:18.312403 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls podName:63243f80-5a2a-4991-91e4-3f217aa9c2cb nodeName:}" failed. No retries permitted until 2026-07-28 16:22:34.312382051 +0000 UTC m=+95.694571663 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls") pod "insights-runtime-extractor-chq49" (UID: "63243f80-5a2a-4991-91e4-3f217aa9c2cb") : secret "insights-runtime-extractor-tls" not found Jul 28 16:22:18.312566 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:18.312527 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/63243f80-5a2a-4991-91e4-3f217aa9c2cb-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:20.101993 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:20.101857 2566 scope.go:117] "RemoveContainer" containerID="dbb3a887a80e30387f9972360f614e13b2cdc1a7b839f5c2168cbf91b2c27ab0" Jul 28 16:22:20.567035 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:20.567006 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:22:20.567384 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:20.567369 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/1.log" Jul 28 16:22:20.567435 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:20.567406 2566 generic.go:356] "Generic (PLEG): container finished" podID="ed0c86cb-26a6-4633-9170-9a042a3a71b3" containerID="7fcee8cdae964d247405a94ac41f3c5235dc70434a5844bae34cb487bf0abb95" exitCode=255 Jul 28 16:22:20.567470 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:20.567438 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" event={"ID":"ed0c86cb-26a6-4633-9170-9a042a3a71b3","Type":"ContainerDied","Data":"7fcee8cdae964d247405a94ac41f3c5235dc70434a5844bae34cb487bf0abb95"} Jul 28 16:22:20.567529 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:20.567517 2566 scope.go:117] "RemoveContainer" containerID="dbb3a887a80e30387f9972360f614e13b2cdc1a7b839f5c2168cbf91b2c27ab0" Jul 28 16:22:20.567835 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:20.567819 2566 scope.go:117] "RemoveContainer" containerID="7fcee8cdae964d247405a94ac41f3c5235dc70434a5844bae34cb487bf0abb95" Jul 28 16:22:20.568038 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:20.568020 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-fcmgz_openshift-console-operator(ed0c86cb-26a6-4633-9170-9a042a3a71b3)\"" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" podUID="ed0c86cb-26a6-4633-9170-9a042a3a71b3" Jul 28 16:22:21.571482 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:21.571454 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:22:28.691174 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:28.691135 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:22:28.691575 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:28.691198 2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:22:28.691575 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:28.691500 2566 scope.go:117] "RemoveContainer" containerID="7fcee8cdae964d247405a94ac41f3c5235dc70434a5844bae34cb487bf0abb95" Jul 28 16:22:28.691700 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:28.691664 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-fcmgz_openshift-console-operator(ed0c86cb-26a6-4633-9170-9a042a3a71b3)\"" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" podUID="ed0c86cb-26a6-4633-9170-9a042a3a71b3" Jul 28 16:22:30.212547 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:30.212511 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fz4bt\" (UID: \"54e904f0-e7ab-4882-9b0d-a91248f578a4\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:22:30.212903 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:30.212594 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:22:30.213202 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:30.213185 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cab67026-e646-4a71-9c1a-e21d1fda8536-service-ca-bundle\") pod \"router-default-855657b557-78fr5\" (UID: \"cab67026-e646-4a71-9c1a-e21d1fda8536\") " pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:22:30.215037 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:30.215017 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/54e904f0-e7ab-4882-9b0d-a91248f578a4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-fz4bt\" (UID: \"54e904f0-e7ab-4882-9b0d-a91248f578a4\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:22:30.407572 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:30.407537 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dpb5t\"" Jul 28 16:22:30.415783 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:30.415760 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" Jul 28 16:22:30.427295 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:30.427271 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j7dqr\"" Jul 28 16:22:30.435617 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:30.435585 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:22:30.579655 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:30.579624 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt"] Jul 28 16:22:30.582949 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:22:30.582905 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54e904f0_e7ab_4882_9b0d_a91248f578a4.slice/crio-ed59745739607c6c5b5fa88958531b844dd575cd5ebda3583855c71b930cf758 WatchSource:0}: Error finding container ed59745739607c6c5b5fa88958531b844dd575cd5ebda3583855c71b930cf758: Status 404 returned error can't find the container with id ed59745739607c6c5b5fa88958531b844dd575cd5ebda3583855c71b930cf758 Jul 28 16:22:30.603536 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:30.603506 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" event={"ID":"54e904f0-e7ab-4882-9b0d-a91248f578a4","Type":"ContainerStarted","Data":"ed59745739607c6c5b5fa88958531b844dd575cd5ebda3583855c71b930cf758"} Jul 28 16:22:30.606939 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:30.606911 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-855657b557-78fr5"] Jul 28 16:22:30.609419 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:22:30.609396 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcab67026_e646_4a71_9c1a_e21d1fda8536.slice/crio-8b4511a52340f6c6d80da0fef780e2b5a24fc2c2e4a7bc6f987402c1b738d393 WatchSource:0}: Error finding container 8b4511a52340f6c6d80da0fef780e2b5a24fc2c2e4a7bc6f987402c1b738d393: Status 404 returned error can't find the container with id 8b4511a52340f6c6d80da0fef780e2b5a24fc2c2e4a7bc6f987402c1b738d393 Jul 28 16:22:31.608894 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:31.608853 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-855657b557-78fr5" event={"ID":"cab67026-e646-4a71-9c1a-e21d1fda8536","Type":"ContainerStarted","Data":"3178bda98bc85220380d8b354468c6c356473592d068da710898e87b321452f4"} Jul 28 16:22:31.609355 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:31.608915 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-855657b557-78fr5" event={"ID":"cab67026-e646-4a71-9c1a-e21d1fda8536","Type":"ContainerStarted","Data":"8b4511a52340f6c6d80da0fef780e2b5a24fc2c2e4a7bc6f987402c1b738d393"} Jul 28 16:22:31.628356 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:31.628304 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-855657b557-78fr5" podStartSLOduration=33.628287947 podStartE2EDuration="33.628287947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:31.627307452 +0000 UTC m=+93.009497070" watchObservedRunningTime="2026-07-28 16:22:31.628287947 +0000 UTC m=+93.010477558" Jul 28 16:22:32.436830 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:32.436790 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:22:32.439906 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:32.439837 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:22:32.614420 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:32.614386 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" event={"ID":"54e904f0-e7ab-4882-9b0d-a91248f578a4","Type":"ContainerStarted","Data":"5344413266e2f3366b2b203e430be99b3d5e5a5a6c7ac8b049348af4b79037f4"} Jul 28 16:22:32.614846 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:32.614543 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:22:32.615843 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:32.615814 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-855657b557-78fr5" Jul 28 16:22:32.655892 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:32.655834 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-fz4bt" podStartSLOduration=33.049152817 podStartE2EDuration="34.655808976s" totalImagesPullingTime="1.606656159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:32.631268363 +0000 UTC m=+94.013457980" watchObservedRunningTime="2026-07-28 16:22:32.655808976 +0000 UTC m=+94.037998649" Jul 28 16:22:32.688095 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:32.688057 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-kds9d"] Jul 28 16:22:32.709714 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:32.709682 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-kds9d"] Jul 28 16:22:32.709846 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:32.709823 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-kds9d" Jul 28 16:22:32.712545 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:32.712523 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-6rdgg\"" Jul 28 16:22:32.712655 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:32.712582 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 28 16:22:32.838148 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:32.838109 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/53c435c8-0bb5-464f-8ef0-39483cd69209-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-kds9d\" (UID: \"53c435c8-0bb5-464f-8ef0-39483cd69209\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-kds9d" Jul 28 16:22:32.938698 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:32.938661 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/53c435c8-0bb5-464f-8ef0-39483cd69209-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-kds9d\" (UID: \"53c435c8-0bb5-464f-8ef0-39483cd69209\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-kds9d" Jul 28 16:22:32.938849 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:32.938761 2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 28 16:22:32.938849 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:32.938832 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53c435c8-0bb5-464f-8ef0-39483cd69209-tls-certificates podName:53c435c8-0bb5-464f-8ef0-39483cd69209 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:33.438815093 +0000 UTC m=+94.821004691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/53c435c8-0bb5-464f-8ef0-39483cd69209-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-kds9d" (UID: "53c435c8-0bb5-464f-8ef0-39483cd69209") : secret "prometheus-operator-admission-webhook-tls" not found Jul 28 16:22:33.443800 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:33.443758 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/53c435c8-0bb5-464f-8ef0-39483cd69209-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-kds9d\" (UID: \"53c435c8-0bb5-464f-8ef0-39483cd69209\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-kds9d" Jul 28 16:22:33.444000 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:33.443919 2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 28 16:22:33.444051 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:33.444006 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53c435c8-0bb5-464f-8ef0-39483cd69209-tls-certificates podName:53c435c8-0bb5-464f-8ef0-39483cd69209 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:34.443991123 +0000 UTC m=+95.826180732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/53c435c8-0bb5-464f-8ef0-39483cd69209-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-kds9d" (UID: "53c435c8-0bb5-464f-8ef0-39483cd69209") : secret "prometheus-operator-admission-webhook-tls" not found Jul 28 16:22:34.351774 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:34.351719 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:34.354194 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:34.354173 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/63243f80-5a2a-4991-91e4-3f217aa9c2cb-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-chq49\" (UID: \"63243f80-5a2a-4991-91e4-3f217aa9c2cb\") " pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:34.452895 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:34.452853 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/53c435c8-0bb5-464f-8ef0-39483cd69209-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-kds9d\" (UID: \"53c435c8-0bb5-464f-8ef0-39483cd69209\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-kds9d" Jul 28 16:22:34.455462 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:34.455439 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/53c435c8-0bb5-464f-8ef0-39483cd69209-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-kds9d\" (UID: \"53c435c8-0bb5-464f-8ef0-39483cd69209\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-kds9d" Jul 28 16:22:34.520408 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:34.520373 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-kds9d" Jul 28 16:22:34.646380 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:34.646333 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-kds9d"] Jul 28 16:22:34.648795 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:22:34.648754 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c435c8_0bb5_464f_8ef0_39483cd69209.slice/crio-4638c6de710a94e3966384726c1eff02491544d7425afdfbba0eebbdf593eefc WatchSource:0}: Error finding container 4638c6de710a94e3966384726c1eff02491544d7425afdfbba0eebbdf593eefc: Status 404 returned error can't find the container with id 4638c6de710a94e3966384726c1eff02491544d7425afdfbba0eebbdf593eefc Jul 28 16:22:34.651051 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:34.651030 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-chq49" Jul 28 16:22:34.777305 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:34.777274 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-chq49"] Jul 28 16:22:34.780014 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:22:34.779988 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63243f80_5a2a_4991_91e4_3f217aa9c2cb.slice/crio-56da71c645e02d6aa3999cc8b53cd6f56258ffc328988622501106bc32f7d89a WatchSource:0}: Error finding container 56da71c645e02d6aa3999cc8b53cd6f56258ffc328988622501106bc32f7d89a: Status 404 returned error can't find the container with id 56da71c645e02d6aa3999cc8b53cd6f56258ffc328988622501106bc32f7d89a Jul 28 16:22:35.628142 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:35.628093 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-chq49" event={"ID":"63243f80-5a2a-4991-91e4-3f217aa9c2cb","Type":"ContainerStarted","Data":"56da71c645e02d6aa3999cc8b53cd6f56258ffc328988622501106bc32f7d89a"} Jul 28 16:22:35.629542 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:35.629516 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-kds9d" event={"ID":"53c435c8-0bb5-464f-8ef0-39483cd69209","Type":"ContainerStarted","Data":"4638c6de710a94e3966384726c1eff02491544d7425afdfbba0eebbdf593eefc"} Jul 28 16:22:36.637580 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:36.637539 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-kds9d" event={"ID":"53c435c8-0bb5-464f-8ef0-39483cd69209","Type":"ContainerStarted","Data":"5b037ed9d0b23739002736b3a839a7afa7f0cace6edfe808dc887cc2980660f2"} Jul 28 16:22:36.638030 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:36.637747 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-kds9d" Jul 28 16:22:36.643305 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:36.643281 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-kds9d" Jul 28 16:22:36.653523 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:36.653469 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-kds9d" podStartSLOduration=3.716337743 podStartE2EDuration="4.653450535s" totalImagesPullingTime="937.112792ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:36.65256987 +0000 UTC m=+98.034759489" watchObservedRunningTime="2026-07-28 16:22:36.653450535 +0000 UTC m=+98.035640154" Jul 28 16:22:37.750651 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.750615 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-cqgvf"] Jul 28 16:22:37.759145 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.759113 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" Jul 28 16:22:37.763183 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.762766 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 28 16:22:37.763183 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.762898 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 28 16:22:37.763183 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.762768 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 28 16:22:37.763183 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.762768 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-s6rtc\"" Jul 28 16:22:37.763183 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.763095 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-cqgvf"] Jul 28 16:22:37.884353 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.884299 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29fdb1e7-dba8-4116-bd0b-a28d038bdf39-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-cqgvf\" (UID: \"29fdb1e7-dba8-4116-bd0b-a28d038bdf39\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" Jul 28 16:22:37.884523 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.884443 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xphv\" (UniqueName: \"kubernetes.io/projected/29fdb1e7-dba8-4116-bd0b-a28d038bdf39-kube-api-access-9xphv\") pod \"prometheus-operator-84d69f794d-cqgvf\" (UID: \"29fdb1e7-dba8-4116-bd0b-a28d038bdf39\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" Jul 28 16:22:37.884523 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.884481 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29fdb1e7-dba8-4116-bd0b-a28d038bdf39-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-cqgvf\" (UID: \"29fdb1e7-dba8-4116-bd0b-a28d038bdf39\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" Jul 28 16:22:37.884523 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.884512 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/29fdb1e7-dba8-4116-bd0b-a28d038bdf39-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-cqgvf\" (UID: \"29fdb1e7-dba8-4116-bd0b-a28d038bdf39\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" Jul 28 16:22:37.985044 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.985002 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/29fdb1e7-dba8-4116-bd0b-a28d038bdf39-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-cqgvf\" (UID: \"29fdb1e7-dba8-4116-bd0b-a28d038bdf39\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" Jul 28 16:22:37.985236 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.985117 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29fdb1e7-dba8-4116-bd0b-a28d038bdf39-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-cqgvf\" (UID: \"29fdb1e7-dba8-4116-bd0b-a28d038bdf39\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" Jul 28 16:22:37.985236 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.985175 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xphv\" (UniqueName: \"kubernetes.io/projected/29fdb1e7-dba8-4116-bd0b-a28d038bdf39-kube-api-access-9xphv\") pod \"prometheus-operator-84d69f794d-cqgvf\" (UID: \"29fdb1e7-dba8-4116-bd0b-a28d038bdf39\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" Jul 28 16:22:37.985236 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.985206 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29fdb1e7-dba8-4116-bd0b-a28d038bdf39-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-cqgvf\" (UID: \"29fdb1e7-dba8-4116-bd0b-a28d038bdf39\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" Jul 28 16:22:37.985953 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.985904 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29fdb1e7-dba8-4116-bd0b-a28d038bdf39-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-cqgvf\" (UID: \"29fdb1e7-dba8-4116-bd0b-a28d038bdf39\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" Jul 28 16:22:37.988183 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.988159 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/29fdb1e7-dba8-4116-bd0b-a28d038bdf39-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-cqgvf\" (UID: \"29fdb1e7-dba8-4116-bd0b-a28d038bdf39\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" Jul 28 16:22:37.988293 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.988222 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/29fdb1e7-dba8-4116-bd0b-a28d038bdf39-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-cqgvf\" (UID: \"29fdb1e7-dba8-4116-bd0b-a28d038bdf39\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" Jul 28 16:22:37.993884 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:37.993858 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xphv\" (UniqueName: \"kubernetes.io/projected/29fdb1e7-dba8-4116-bd0b-a28d038bdf39-kube-api-access-9xphv\") pod \"prometheus-operator-84d69f794d-cqgvf\" (UID: \"29fdb1e7-dba8-4116-bd0b-a28d038bdf39\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" Jul 28 16:22:38.072680 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:38.072605 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" Jul 28 16:22:38.489674 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:38.489642 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:22:38.489883 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:38.489724 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:22:38.492402 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:38.492374 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/08c6af7a-3f2a-491b-aee3-61a1dafe2c0a-metrics-tls\") pod \"dns-default-fdhkh\" (UID: \"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a\") " pod="openshift-dns/dns-default-fdhkh" Jul 28 16:22:38.492630 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:38.492611 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls\") pod \"image-registry-554f97cd94-4klh7\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:22:38.760848 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:38.760765 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zshnb\"" Jul 28 16:22:38.769043 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:38.769021 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:22:38.775873 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:38.775846 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hmx4q\"" Jul 28 16:22:38.783809 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:38.783782 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fdhkh" Jul 28 16:22:40.206353 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:40.205810 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-cqgvf"] Jul 28 16:22:40.208165 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:22:40.208132 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29fdb1e7_dba8_4116_bd0b_a28d038bdf39.slice/crio-e254add7ae0b5442352726076926aa02d3953f042df966ca32d580121dff412f WatchSource:0}: Error finding container e254add7ae0b5442352726076926aa02d3953f042df966ca32d580121dff412f: Status 404 returned error can't find the container with id e254add7ae0b5442352726076926aa02d3953f042df966ca32d580121dff412f Jul 28 16:22:40.433281 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:40.433251 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-554f97cd94-4klh7"] Jul 28 16:22:40.434551 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:40.434528 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fdhkh"] Jul 28 16:22:40.436094 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:22:40.436054 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbae9c53c_7ff2_43b0_b0ac_5433a37338e4.slice/crio-08df79612230aa572c0c27abd66928867e61308bea8b12058fc0ce3b331184a5 WatchSource:0}: Error finding container 08df79612230aa572c0c27abd66928867e61308bea8b12058fc0ce3b331184a5: Status 404 returned error can't find the container with id 08df79612230aa572c0c27abd66928867e61308bea8b12058fc0ce3b331184a5 Jul 28 16:22:40.437571 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:22:40.437548 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c6af7a_3f2a_491b_aee3_61a1dafe2c0a.slice/crio-18d77d8afc1c00df3688e896418ce778a34b5ff8e5bbb440c0a16c8dc1be7101 WatchSource:0}: Error finding container 18d77d8afc1c00df3688e896418ce778a34b5ff8e5bbb440c0a16c8dc1be7101: Status 404 returned error can't find the container with id 18d77d8afc1c00df3688e896418ce778a34b5ff8e5bbb440c0a16c8dc1be7101 Jul 28 16:22:40.654162 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:40.654068 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fdhkh" event={"ID":"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a","Type":"ContainerStarted","Data":"18d77d8afc1c00df3688e896418ce778a34b5ff8e5bbb440c0a16c8dc1be7101"} Jul 28 16:22:40.655354 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:40.655310 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" event={"ID":"29fdb1e7-dba8-4116-bd0b-a28d038bdf39","Type":"ContainerStarted","Data":"e254add7ae0b5442352726076926aa02d3953f042df966ca32d580121dff412f"} Jul 28 16:22:40.656809 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:40.656780 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-chq49" event={"ID":"63243f80-5a2a-4991-91e4-3f217aa9c2cb","Type":"ContainerStarted","Data":"135c9118b409da74edf68d8d237d2854dd3fbf757fcb341260e5e0df49d808b3"} Jul 28 16:22:40.658106 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:40.658083 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-554f97cd94-4klh7" event={"ID":"bae9c53c-7ff2-43b0-b0ac-5433a37338e4","Type":"ContainerStarted","Data":"5e2807d83e725e9381a47ce9a140d77dfbd2ac41a77ba2ce2db1d1f6afd5ba95"} Jul 28 16:22:40.658184 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:40.658121 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-554f97cd94-4klh7" event={"ID":"bae9c53c-7ff2-43b0-b0ac-5433a37338e4","Type":"ContainerStarted","Data":"08df79612230aa572c0c27abd66928867e61308bea8b12058fc0ce3b331184a5"} Jul 28 16:22:40.658226 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:40.658220 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:22:40.677402 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:40.677338 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-554f97cd94-4klh7" podStartSLOduration=100.677315787 podStartE2EDuration="1m40.677315787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:40.676620419 +0000 UTC m=+102.058810039" watchObservedRunningTime="2026-07-28 16:22:40.677315787 +0000 UTC m=+102.059505408" Jul 28 16:22:41.539044 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:41.538928 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gbdb4" Jul 28 16:22:42.103353 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:42.103265 2566 scope.go:117] "RemoveContainer" containerID="7fcee8cdae964d247405a94ac41f3c5235dc70434a5844bae34cb487bf0abb95" Jul 28 16:22:42.671350 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:42.671301 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" event={"ID":"29fdb1e7-dba8-4116-bd0b-a28d038bdf39","Type":"ContainerStarted","Data":"bd225cd8e5a6c9cae68d2c78c445d83fad9f14302b865b09a8a77081df0bb350"} Jul 28 16:22:42.671799 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:42.671419 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" event={"ID":"29fdb1e7-dba8-4116-bd0b-a28d038bdf39","Type":"ContainerStarted","Data":"e69b754d8c40c8312e00c9ebc99a6007dd2b6c4fa3cec54fb1fc81d255dff48c"} Jul 28 16:22:43.678130 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:43.678099 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:22:43.678595 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:43.678238 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" event={"ID":"ed0c86cb-26a6-4633-9170-9a042a3a71b3","Type":"ContainerStarted","Data":"0e6986794e2629a2aa506a0362afd5d9c48c32c4ca3612e69e135cbc94e77bb1"} Jul 28 16:22:43.698724 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:43.698644 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" podStartSLOduration=42.626176349 podStartE2EDuration="45.698619033s" totalImagesPullingTime="3.072442684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:43.698330386 +0000 UTC m=+105.080520030" watchObservedRunningTime="2026-07-28 16:22:43.698619033 +0000 UTC m=+105.080808651" Jul 28 16:22:43.699036 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:43.698818 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-cqgvf" podStartSLOduration=5.053545889 podStartE2EDuration="6.698807216s" totalImagesPullingTime="1.645261327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:42.688825426 +0000 UTC m=+104.071015045" watchObservedRunningTime="2026-07-28 16:22:43.698807216 +0000 UTC m=+105.080996812" Jul 28 16:22:44.165000 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.164944 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ts6kf"] Jul 28 16:22:44.181054 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.181018 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.184101 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.184071 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 28 16:22:44.185990 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.185915 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 28 16:22:44.185990 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.185936 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 28 16:22:44.186170 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.186026 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bqvfh\"" Jul 28 16:22:44.239352 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.239318 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-accelerators-collector-config\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.239565 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.239364 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19611934-daaf-47b3-b63f-2159cb2d7f69-metrics-client-ca\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.239565 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.239392 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.239770 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.239747 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2bdj\" (UniqueName: \"kubernetes.io/projected/19611934-daaf-47b3-b63f-2159cb2d7f69-kube-api-access-z2bdj\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.239932 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.239812 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/19611934-daaf-47b3-b63f-2159cb2d7f69-root\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.239932 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.239841 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-tls\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.239932 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.239869 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19611934-daaf-47b3-b63f-2159cb2d7f69-sys\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.239932 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.239923 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-wtmp\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.240117 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.239953 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-textfile\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.341136 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.341103 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2bdj\" (UniqueName: \"kubernetes.io/projected/19611934-daaf-47b3-b63f-2159cb2d7f69-kube-api-access-z2bdj\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.341330 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.341178 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/19611934-daaf-47b3-b63f-2159cb2d7f69-root\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.341330 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.341195 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-tls\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.341330 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.341221 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19611934-daaf-47b3-b63f-2159cb2d7f69-sys\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.341330 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.341252 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-wtmp\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.341330 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.341283 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-textfile\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.341330 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.341299 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/19611934-daaf-47b3-b63f-2159cb2d7f69-root\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.341633 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.341354 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19611934-daaf-47b3-b63f-2159cb2d7f69-sys\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.341633 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:44.341452 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 28 16:22:44.341633 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.341481 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-accelerators-collector-config\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.341633 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:44.341513 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-tls podName:19611934-daaf-47b3-b63f-2159cb2d7f69 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:44.841493596 +0000 UTC m=+106.223683195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-tls") pod "node-exporter-ts6kf" (UID: "19611934-daaf-47b3-b63f-2159cb2d7f69") : secret "node-exporter-tls" not found Jul 28 16:22:44.341633 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.341448 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-wtmp\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.341633 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.341554 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19611934-daaf-47b3-b63f-2159cb2d7f69-metrics-client-ca\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.341633 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.341581 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.341633 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.341596 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-textfile\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.342066 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.342051 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19611934-daaf-47b3-b63f-2159cb2d7f69-metrics-client-ca\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.342127 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.342076 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-accelerators-collector-config\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.344288 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.344266 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.350788 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.350756 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2bdj\" (UniqueName: \"kubernetes.io/projected/19611934-daaf-47b3-b63f-2159cb2d7f69-kube-api-access-z2bdj\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.846355 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.846313 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-tls\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:44.849126 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:44.849097 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/19611934-daaf-47b3-b63f-2159cb2d7f69-node-exporter-tls\") pod \"node-exporter-ts6kf\" (UID: \"19611934-daaf-47b3-b63f-2159cb2d7f69\") " pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:45.091816 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.091718 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ts6kf" Jul 28 16:22:45.152819 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:22:45.152792 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19611934_daaf_47b3_b63f_2159cb2d7f69.slice/crio-72a42e58dcec7abbcd28a262901c04802ef31d4a7f42e1e294ba3a21ca2fc1f9 WatchSource:0}: Error finding container 72a42e58dcec7abbcd28a262901c04802ef31d4a7f42e1e294ba3a21ca2fc1f9: Status 404 returned error can't find the container with id 72a42e58dcec7abbcd28a262901c04802ef31d4a7f42e1e294ba3a21ca2fc1f9 Jul 28 16:22:45.226270 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.226226 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:22:45.230199 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.230178 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.233905 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.233760 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 16:22:45.233905 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.233897 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 16:22:45.234169 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.234033 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 16:22:45.234169 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.234075 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fq657\"" Jul 28 16:22:45.234271 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.234170 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 16:22:45.234271 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.234211 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 16:22:45.234374 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.234276 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 16:22:45.234429 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.234384 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 16:22:45.234429 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.234392 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 16:22:45.234516 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.234461 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 16:22:45.249589 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.249561 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.249736 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.249605 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.249736 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.249636 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwkph\" (UniqueName: \"kubernetes.io/projected/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-kube-api-access-cwkph\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.249736 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.249663 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.249736 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.249691 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-config-volume\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.249736 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.249721 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.250127 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.249749 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-web-config\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.250127 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.249797 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.250127 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.249836 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.250127 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.249868 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.250127 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.249904 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-config-out\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.250127 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.249934 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.250575 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.250174 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.252355 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.252332 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:22:45.350836 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.350796 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwkph\" (UniqueName: \"kubernetes.io/projected/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-kube-api-access-cwkph\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.350964 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.350847 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.350964 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.350877 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-config-volume\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.350964 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.350908 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.350964 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.350933 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-web-config\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.350964 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.351003 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.351329 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.351028 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.351329 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.351056 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.351329 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.351092 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-config-out\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.351329 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.351120 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.351329 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.351151 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.351329 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.351245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.351329 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.351275 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.352162 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:45.351879 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-alertmanager-trusted-ca-bundle podName:f1553d0e-49bb-4ee5-8ddd-cfa991b34745 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:45.851856433 +0000 UTC m=+107.234046037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "f1553d0e-49bb-4ee5-8ddd-cfa991b34745") : configmap references non-existent config key: ca-bundle.crt Jul 28 16:22:45.352576 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.351277 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.352576 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:45.352513 2566 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 28 16:22:45.353066 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:22:45.352681 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-main-tls podName:f1553d0e-49bb-4ee5-8ddd-cfa991b34745 nodeName:}" failed. No retries permitted until 2026-07-28 16:22:45.85255696 +0000 UTC m=+107.234746571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "f1553d0e-49bb-4ee5-8ddd-cfa991b34745") : secret "alertmanager-main-tls" not found Jul 28 16:22:45.353066 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.353023 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.356295 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.356246 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.356295 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.356271 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.356999 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.356935 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.357499 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.357108 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.357499 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.357320 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-config-out\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.357499 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.357459 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-web-config\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.359759 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.359710 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.360139 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.360117 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-config-volume\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.364052 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.363967 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwkph\" (UniqueName: \"kubernetes.io/projected/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-kube-api-access-cwkph\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.689684 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.689639 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-chq49" event={"ID":"63243f80-5a2a-4991-91e4-3f217aa9c2cb","Type":"ContainerStarted","Data":"b7b1fec35e0285e1e61b6b1d7ce0edaeb530a2ecf5a62df590bf54a903db83a7"} Jul 28 16:22:45.693019 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.692905 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fdhkh" event={"ID":"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a","Type":"ContainerStarted","Data":"c34f6baba5fd1cdeae4a16c4a2c9490ad23e84af700150da195cbefc587086b9"} Jul 28 16:22:45.693019 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.692962 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fdhkh" event={"ID":"08c6af7a-3f2a-491b-aee3-61a1dafe2c0a","Type":"ContainerStarted","Data":"c1ef259da7a7da5fa92124e050051d03550b3584c83b1594e51377b563cad9b9"} Jul 28 16:22:45.693318 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.693267 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fdhkh" Jul 28 16:22:45.695101 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.695047 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ts6kf" event={"ID":"19611934-daaf-47b3-b63f-2159cb2d7f69","Type":"ContainerStarted","Data":"72a42e58dcec7abbcd28a262901c04802ef31d4a7f42e1e294ba3a21ca2fc1f9"} Jul 28 16:22:45.712463 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.712316 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fdhkh" podStartSLOduration=67.003234615 podStartE2EDuration="1m11.712297306s" totalImagesPullingTime="4.709062691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:45.710092049 +0000 UTC m=+107.092281666" watchObservedRunningTime="2026-07-28 16:22:45.712297306 +0000 UTC m=+107.094486924" Jul 28 16:22:45.858827 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.858790 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.859558 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.858862 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.859694 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.859651 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:45.867562 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:45.867531 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:46.142698 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:46.142653 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:46.434843 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:46.434812 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:22:46.438405 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:22:46.438365 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1553d0e_49bb_4ee5_8ddd_cfa991b34745.slice/crio-1f8f93974862c8a945517b01f97558ec9e7b7b0fd983d4ed15e592d0ccfa948c WatchSource:0}: Error finding container 1f8f93974862c8a945517b01f97558ec9e7b7b0fd983d4ed15e592d0ccfa948c: Status 404 returned error can't find the container with id 1f8f93974862c8a945517b01f97558ec9e7b7b0fd983d4ed15e592d0ccfa948c Jul 28 16:22:46.701329 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:46.701240 2566 generic.go:356] "Generic (PLEG): container finished" podID="19611934-daaf-47b3-b63f-2159cb2d7f69" containerID="e3f5d82eb2a3df76f495f30dcec3d7ce3028e0945294bc7dd756d8f37a3a374c" exitCode=0 Jul 28 16:22:46.701497 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:46.701367 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ts6kf" event={"ID":"19611934-daaf-47b3-b63f-2159cb2d7f69","Type":"ContainerDied","Data":"e3f5d82eb2a3df76f495f30dcec3d7ce3028e0945294bc7dd756d8f37a3a374c"} Jul 28 16:22:46.703652 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:46.703623 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1553d0e-49bb-4ee5-8ddd-cfa991b34745","Type":"ContainerStarted","Data":"1f8f93974862c8a945517b01f97558ec9e7b7b0fd983d4ed15e592d0ccfa948c"} Jul 28 16:22:47.710226 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:47.710191 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ts6kf" event={"ID":"19611934-daaf-47b3-b63f-2159cb2d7f69","Type":"ContainerStarted","Data":"81830d489ac07c112025a78860c4ae17ee4e33c9bef8f0ba2ab4e41f6245cbb3"} Jul 28 16:22:47.710706 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:47.710254 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ts6kf" event={"ID":"19611934-daaf-47b3-b63f-2159cb2d7f69","Type":"ContainerStarted","Data":"136904e31fe6aaf0c8d40ce0a747b97abbe441b41ddf272c1e279e50a3f49804"} Jul 28 16:22:47.731559 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:47.731318 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ts6kf" podStartSLOduration=2.523871911 podStartE2EDuration="3.731294083s" totalImagesPullingTime="1.116729144s" totalInitContainerRuntime="90.693028ms" isStatefulPod=true podCreationTimestamp="2026-07-28 16:22:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:47.728263654 +0000 UTC m=+109.110453267" watchObservedRunningTime="2026-07-28 16:22:47.731294083 +0000 UTC m=+109.113483702" Jul 28 16:22:48.547960 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.547925 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7b9878b988-mgx2k"] Jul 28 16:22:48.557124 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.557096 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.561233 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.560928 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 28 16:22:48.561385 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.561252 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-439i89q3e9hs7\"" Jul 28 16:22:48.561600 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.561571 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 28 16:22:48.561751 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.561663 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dp76c\"" Jul 28 16:22:48.562051 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.561597 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 28 16:22:48.562796 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.562774 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 28 16:22:48.564782 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.564758 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b9878b988-mgx2k"] Jul 28 16:22:48.585192 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.585163 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0020da7d-430b-4e21-9acb-737c6e75b06e-secret-metrics-server-tls\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.585192 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.585206 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0020da7d-430b-4e21-9acb-737c6e75b06e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.585396 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.585234 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0020da7d-430b-4e21-9acb-737c6e75b06e-client-ca-bundle\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.585396 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.585258 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0020da7d-430b-4e21-9acb-737c6e75b06e-secret-metrics-server-client-certs\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.585396 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.585356 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0020da7d-430b-4e21-9acb-737c6e75b06e-audit-log\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.585515 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.585413 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j59t7\" (UniqueName: \"kubernetes.io/projected/0020da7d-430b-4e21-9acb-737c6e75b06e-kube-api-access-j59t7\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.585515 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.585462 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0020da7d-430b-4e21-9acb-737c6e75b06e-metrics-server-audit-profiles\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.686455 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.686419 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0020da7d-430b-4e21-9acb-737c6e75b06e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.686715 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.686462 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0020da7d-430b-4e21-9acb-737c6e75b06e-client-ca-bundle\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.686715 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.686493 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0020da7d-430b-4e21-9acb-737c6e75b06e-secret-metrics-server-client-certs\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.686715 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.686544 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0020da7d-430b-4e21-9acb-737c6e75b06e-audit-log\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.686715 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.686581 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j59t7\" (UniqueName: \"kubernetes.io/projected/0020da7d-430b-4e21-9acb-737c6e75b06e-kube-api-access-j59t7\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.686715 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.686613 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0020da7d-430b-4e21-9acb-737c6e75b06e-metrics-server-audit-profiles\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.686715 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.686680 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0020da7d-430b-4e21-9acb-737c6e75b06e-secret-metrics-server-tls\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.687231 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.687201 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0020da7d-430b-4e21-9acb-737c6e75b06e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.687567 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.687543 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0020da7d-430b-4e21-9acb-737c6e75b06e-audit-log\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.688333 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.688302 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0020da7d-430b-4e21-9acb-737c6e75b06e-metrics-server-audit-profiles\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.691055 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.691033 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0020da7d-430b-4e21-9acb-737c6e75b06e-client-ca-bundle\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.691282 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.691217 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0020da7d-430b-4e21-9acb-737c6e75b06e-secret-metrics-server-client-certs\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.691282 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.691236 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0020da7d-430b-4e21-9acb-737c6e75b06e-secret-metrics-server-tls\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.696759 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.696717 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j59t7\" (UniqueName: \"kubernetes.io/projected/0020da7d-430b-4e21-9acb-737c6e75b06e-kube-api-access-j59t7\") pod \"metrics-server-7b9878b988-mgx2k\" (UID: \"0020da7d-430b-4e21-9acb-737c6e75b06e\") " pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.869648 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.869564 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:22:48.895509 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.895475 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-j5jn5"] Jul 28 16:22:48.903056 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.903029 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j5jn5" Jul 28 16:22:48.905485 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.905468 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 28 16:22:48.905600 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.905586 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-fxwvt\"" Jul 28 16:22:48.908544 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.908519 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-j5jn5"] Jul 28 16:22:48.988731 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.988627 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss8nl\" (UniqueName: \"kubernetes.io/projected/179cd673-52a8-4c1d-97e4-88ee48ac0483-kube-api-access-ss8nl\") pod \"monitoring-plugin-68b7fc9574-j5jn5\" (UID: \"179cd673-52a8-4c1d-97e4-88ee48ac0483\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j5jn5" Jul 28 16:22:48.988731 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:48.988731 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/179cd673-52a8-4c1d-97e4-88ee48ac0483-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-j5jn5\" (UID: \"179cd673-52a8-4c1d-97e4-88ee48ac0483\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j5jn5" Jul 28 16:22:49.089513 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:49.089477 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/179cd673-52a8-4c1d-97e4-88ee48ac0483-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-j5jn5\" (UID: \"179cd673-52a8-4c1d-97e4-88ee48ac0483\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j5jn5" Jul 28 16:22:49.089719 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:49.089585 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss8nl\" (UniqueName: \"kubernetes.io/projected/179cd673-52a8-4c1d-97e4-88ee48ac0483-kube-api-access-ss8nl\") pod \"monitoring-plugin-68b7fc9574-j5jn5\" (UID: \"179cd673-52a8-4c1d-97e4-88ee48ac0483\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j5jn5" Jul 28 16:22:49.092496 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:49.092470 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/179cd673-52a8-4c1d-97e4-88ee48ac0483-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-j5jn5\" (UID: \"179cd673-52a8-4c1d-97e4-88ee48ac0483\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j5jn5" Jul 28 16:22:49.097759 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:49.097735 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss8nl\" (UniqueName: \"kubernetes.io/projected/179cd673-52a8-4c1d-97e4-88ee48ac0483-kube-api-access-ss8nl\") pod \"monitoring-plugin-68b7fc9574-j5jn5\" (UID: \"179cd673-52a8-4c1d-97e4-88ee48ac0483\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j5jn5" Jul 28 16:22:49.216462 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:49.216382 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j5jn5" Jul 28 16:22:49.419997 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:49.419953 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-j5jn5"] Jul 28 16:22:49.436846 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:49.436769 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7b9878b988-mgx2k"] Jul 28 16:22:49.720552 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:49.720517 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-chq49" event={"ID":"63243f80-5a2a-4991-91e4-3f217aa9c2cb","Type":"ContainerStarted","Data":"2d7949f2379ffc8fa37ce743ae93cac838989b2be31b1efec6d886e68e5c2647"} Jul 28 16:22:49.721752 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:49.721728 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j5jn5" event={"ID":"179cd673-52a8-4c1d-97e4-88ee48ac0483","Type":"ContainerStarted","Data":"12f78ff8844d524a55489b2f3d39ee887ca51eee9071105fb3a13702f357bc12"} Jul 28 16:22:49.722927 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:49.722904 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" event={"ID":"0020da7d-430b-4e21-9acb-737c6e75b06e","Type":"ContainerStarted","Data":"f0c8da6e0ef0e9318671f05f99aa402d3842f6df27b73e1884cb3a9131240f88"} Jul 28 16:22:49.724504 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:49.724481 2566 generic.go:356] "Generic (PLEG): container finished" podID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerID="284cb63b01d65a3f19eae9ed14bff5e32151863e5f879927ef212635bbc7e65d" exitCode=0 Jul 28 16:22:49.724590 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:49.724515 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1553d0e-49bb-4ee5-8ddd-cfa991b34745","Type":"ContainerDied","Data":"284cb63b01d65a3f19eae9ed14bff5e32151863e5f879927ef212635bbc7e65d"} Jul 28 16:22:49.738587 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:49.738548 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-chq49" podStartSLOduration=33.40192445 podStartE2EDuration="47.738535161s" totalImagesPullingTime="14.336610711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:22:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:49.737021241 +0000 UTC m=+111.119210858" watchObservedRunningTime="2026-07-28 16:22:49.738535161 +0000 UTC m=+111.120724791" Jul 28 16:22:50.402193 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.399893 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 16:22:50.404899 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.404875 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.411287 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.409964 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 28 16:22:50.411287 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.410130 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 28 16:22:50.411287 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.410232 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-m51utjhoeb6t\"" Jul 28 16:22:50.411287 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.410417 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 28 16:22:50.411287 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.410635 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 28 16:22:50.411287 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.410667 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rvgww\"" Jul 28 16:22:50.411287 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.410131 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 28 16:22:50.411287 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.410902 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 28 16:22:50.412159 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.412098 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 28 16:22:50.412159 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.412145 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 28 16:22:50.412462 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.412328 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 28 16:22:50.412680 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.412098 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 28 16:22:50.414226 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.414203 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 28 16:22:50.415953 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.414757 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 28 16:22:50.426009 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.425967 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 16:22:50.429262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.429235 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 28 16:22:50.442226 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.441164 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 28 16:22:50.505091 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505050 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505283 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505113 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505283 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505148 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505283 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505177 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505283 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505202 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-web-config\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505283 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505220 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t82xj\" (UniqueName: \"kubernetes.io/projected/fd11b80b-529d-4497-bafb-0293f59966f6-kube-api-access-t82xj\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505283 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505238 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505283 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505262 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505663 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505341 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fd11b80b-529d-4497-bafb-0293f59966f6-config-out\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505663 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505374 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505663 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505451 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-config\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505663 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505488 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fd11b80b-529d-4497-bafb-0293f59966f6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505663 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505542 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505663 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505576 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505663 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505596 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505663 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505613 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505663 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505639 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.505663 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505663 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.506177 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505687 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.506177 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.505715 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.606613 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.606673 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.606719 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.606755 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.606784 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-web-config\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.606821 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t82xj\" (UniqueName: \"kubernetes.io/projected/fd11b80b-529d-4497-bafb-0293f59966f6-kube-api-access-t82xj\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.606862 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.606887 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.606955 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fd11b80b-529d-4497-bafb-0293f59966f6-config-out\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.607007 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.607043 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-config\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.607073 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fd11b80b-529d-4497-bafb-0293f59966f6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.607120 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.607161 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.607190 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.607214 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.607378 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.607244 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.611785 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.607275 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.611785 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.607302 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.611785 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.607339 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.611785 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.608164 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.611785 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.608319 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.611785 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.609053 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.611785 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.609660 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.611785 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.611279 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.611785 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.611580 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.615553 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.615526 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fd11b80b-529d-4497-bafb-0293f59966f6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.616445 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.616388 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.616550 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.616448 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.616856 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.616823 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-web-config\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.620012 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.618880 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.620012 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.618891 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fd11b80b-529d-4497-bafb-0293f59966f6-config-out\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.620012 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.619423 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.620012 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.619867 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-config\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.620295 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.620236 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.620799 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.620752 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.622184 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.621479 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t82xj\" (UniqueName: \"kubernetes.io/projected/fd11b80b-529d-4497-bafb-0293f59966f6-kube-api-access-t82xj\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.622184 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.622134 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.624614 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.624573 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.628776 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.628726 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:50.744169 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:50.744122 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:22:52.028873 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:52.028847 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 16:22:52.030636 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:22:52.030611 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd11b80b_529d_4497_bafb_0293f59966f6.slice/crio-ace4c84a397e60f5551f6a6844232cf52db2ffb41e3c288051c008ad646f1a1a WatchSource:0}: Error finding container ace4c84a397e60f5551f6a6844232cf52db2ffb41e3c288051c008ad646f1a1a: Status 404 returned error can't find the container with id ace4c84a397e60f5551f6a6844232cf52db2ffb41e3c288051c008ad646f1a1a Jul 28 16:22:52.739345 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:52.739303 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" event={"ID":"0020da7d-430b-4e21-9acb-737c6e75b06e","Type":"ContainerStarted","Data":"fb949cf813fa4c27b8560baf9bd45a4345614b36e3bdd858b274d26be23a3527"} Jul 28 16:22:52.740590 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:52.740567 2566 generic.go:356] "Generic (PLEG): container finished" podID="fd11b80b-529d-4497-bafb-0293f59966f6" containerID="54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052" exitCode=0 Jul 28 16:22:52.740685 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:52.740657 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd11b80b-529d-4497-bafb-0293f59966f6","Type":"ContainerDied","Data":"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052"} Jul 28 16:22:52.740742 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:52.740729 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd11b80b-529d-4497-bafb-0293f59966f6","Type":"ContainerStarted","Data":"ace4c84a397e60f5551f6a6844232cf52db2ffb41e3c288051c008ad646f1a1a"} Jul 28 16:22:52.743794 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:52.743772 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1553d0e-49bb-4ee5-8ddd-cfa991b34745","Type":"ContainerStarted","Data":"d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0"} Jul 28 16:22:52.743889 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:52.743875 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1553d0e-49bb-4ee5-8ddd-cfa991b34745","Type":"ContainerStarted","Data":"b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576"} Jul 28 16:22:52.743935 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:52.743907 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1553d0e-49bb-4ee5-8ddd-cfa991b34745","Type":"ContainerStarted","Data":"1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589"} Jul 28 16:22:52.743935 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:52.743931 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1553d0e-49bb-4ee5-8ddd-cfa991b34745","Type":"ContainerStarted","Data":"3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b"} Jul 28 16:22:52.744022 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:52.743953 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1553d0e-49bb-4ee5-8ddd-cfa991b34745","Type":"ContainerStarted","Data":"0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b"} Jul 28 16:22:52.745209 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:52.745189 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j5jn5" event={"ID":"179cd673-52a8-4c1d-97e4-88ee48ac0483","Type":"ContainerStarted","Data":"2e117b49d2c86f496e6cde291c836c704dc46d9b75f20b7762e5d65c77df5a1b"} Jul 28 16:22:52.745396 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:52.745380 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j5jn5" Jul 28 16:22:52.750166 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:52.750149 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j5jn5" Jul 28 16:22:52.766734 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:52.766692 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" podStartSLOduration=2.363478194 podStartE2EDuration="4.766677776s" totalImagesPullingTime="2.403199582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:52.765221939 +0000 UTC m=+114.147411556" watchObservedRunningTime="2026-07-28 16:22:52.766677776 +0000 UTC m=+114.148867751" Jul 28 16:22:52.781880 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:52.781833 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j5jn5" podStartSLOduration=2.376010263 podStartE2EDuration="4.78181627s" totalImagesPullingTime="2.405806007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:52.779916381 +0000 UTC m=+114.162106001" watchObservedRunningTime="2026-07-28 16:22:52.78181627 +0000 UTC m=+114.164005888" Jul 28 16:22:52.805030 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:52.804998 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-554f97cd94-4klh7"] Jul 28 16:22:53.679319 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:53.679283 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:22:53.683323 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:53.683301 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-fcmgz" Jul 28 16:22:53.755384 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:53.755346 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1553d0e-49bb-4ee5-8ddd-cfa991b34745","Type":"ContainerStarted","Data":"becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3"} Jul 28 16:22:53.784259 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:53.784205 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.013439387 podStartE2EDuration="8.784188332s" totalImagesPullingTime="5.745842387s" totalInitContainerRuntime="24.906558ms" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:53.781661679 +0000 UTC m=+115.163851326" watchObservedRunningTime="2026-07-28 16:22:53.784188332 +0000 UTC m=+115.166377950" Jul 28 16:22:55.706274 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:55.706244 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fdhkh" Jul 28 16:22:56.770699 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:56.770665 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd11b80b-529d-4497-bafb-0293f59966f6","Type":"ContainerStarted","Data":"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb"} Jul 28 16:22:56.771181 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:56.770730 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd11b80b-529d-4497-bafb-0293f59966f6","Type":"ContainerStarted","Data":"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9"} Jul 28 16:22:58.782318 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:58.782282 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd11b80b-529d-4497-bafb-0293f59966f6","Type":"ContainerStarted","Data":"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d"} Jul 28 16:22:58.782809 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:58.782377 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd11b80b-529d-4497-bafb-0293f59966f6","Type":"ContainerStarted","Data":"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff"} Jul 28 16:22:58.782809 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:58.782406 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd11b80b-529d-4497-bafb-0293f59966f6","Type":"ContainerStarted","Data":"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003"} Jul 28 16:22:58.782809 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:58.782431 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd11b80b-529d-4497-bafb-0293f59966f6","Type":"ContainerStarted","Data":"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9"} Jul 28 16:22:58.810402 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:22:58.810354 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.951033168 podStartE2EDuration="8.810336492s" totalImagesPullingTime="4.833875307s" totalInitContainerRuntime="25.428017ms" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:58.807967459 +0000 UTC m=+120.190157079" watchObservedRunningTime="2026-07-28 16:22:58.810336492 +0000 UTC m=+120.192526109" Jul 28 16:23:00.744654 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:00.744612 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:23:02.812132 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:02.812101 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:23:08.869862 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:08.869826 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:23:08.870281 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:08.869916 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:23:08.875639 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:08.875612 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs\") pod \"network-metrics-daemon-gb4cw\" (UID: \"4a8f2962-0386-4066-a462-445cbb6c3f09\") " pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:23:08.878114 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:08.878094 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4a8f2962-0386-4066-a462-445cbb6c3f09-metrics-certs\") pod \"network-metrics-daemon-gb4cw\" (UID: \"4a8f2962-0386-4066-a462-445cbb6c3f09\") " pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:23:09.130440 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:09.130358 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nr6mp\"" Jul 28 16:23:09.138437 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:09.138406 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gb4cw" Jul 28 16:23:09.282278 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:09.282248 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gb4cw"] Jul 28 16:23:09.284238 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:23:09.284209 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a8f2962_0386_4066_a462_445cbb6c3f09.slice/crio-f4b18c5135e8717f84da2ab86f85ec3c7f5c1b119b36747630285372e974a667 WatchSource:0}: Error finding container f4b18c5135e8717f84da2ab86f85ec3c7f5c1b119b36747630285372e974a667: Status 404 returned error can't find the container with id f4b18c5135e8717f84da2ab86f85ec3c7f5c1b119b36747630285372e974a667 Jul 28 16:23:09.827334 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:09.827291 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gb4cw" event={"ID":"4a8f2962-0386-4066-a462-445cbb6c3f09","Type":"ContainerStarted","Data":"f4b18c5135e8717f84da2ab86f85ec3c7f5c1b119b36747630285372e974a667"} Jul 28 16:23:10.832768 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:10.832727 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gb4cw" event={"ID":"4a8f2962-0386-4066-a462-445cbb6c3f09","Type":"ContainerStarted","Data":"5a91076036e08af70e241d4e1de8c47dd7451ec2114a70c817d34adb10647e89"} Jul 28 16:23:10.833153 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:10.832828 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gb4cw" event={"ID":"4a8f2962-0386-4066-a462-445cbb6c3f09","Type":"ContainerStarted","Data":"e05e804163f5506b723620063dcb53f4350ad93d56c51543ce9143975716bf7b"} Jul 28 16:23:10.848437 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:10.848285 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gb4cw" podStartSLOduration=129.95143556 podStartE2EDuration="2m10.848267751s" totalImagesPullingTime="896.832191ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:23:10.847653907 +0000 UTC m=+132.229843525" watchObservedRunningTime="2026-07-28 16:23:10.848267751 +0000 UTC m=+132.230457367" Jul 28 16:23:17.830355 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:17.830289 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-554f97cd94-4klh7" podUID="bae9c53c-7ff2-43b0-b0ac-5433a37338e4" containerName="registry" containerID="cri-o://5e2807d83e725e9381a47ce9a140d77dfbd2ac41a77ba2ce2db1d1f6afd5ba95" gracePeriod=30 Jul 28 16:23:17.865842 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:17.864318 2566 generic.go:356] "Generic (PLEG): container finished" podID="676a1e19-6f6c-42a4-881c-730a584c1c2b" containerID="2d016f4bc80dc2d28ca87b90b6402b08276da08b01ecdbf6bb6162a81f657c7d" exitCode=0 Jul 28 16:23:17.865842 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:17.864416 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-9hwqx" event={"ID":"676a1e19-6f6c-42a4-881c-730a584c1c2b","Type":"ContainerDied","Data":"2d016f4bc80dc2d28ca87b90b6402b08276da08b01ecdbf6bb6162a81f657c7d"} Jul 28 16:23:17.865842 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:17.864812 2566 scope.go:117] "RemoveContainer" containerID="2d016f4bc80dc2d28ca87b90b6402b08276da08b01ecdbf6bb6162a81f657c7d" Jul 28 16:23:18.085211 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.085156 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:23:18.158262 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.158230 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-installation-pull-secrets\") pod \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " Jul 28 16:23:18.158428 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.158279 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-bound-sa-token\") pod \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " Jul 28 16:23:18.158428 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.158313 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-certificates\") pod \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " Jul 28 16:23:18.158428 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.158337 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls\") pod \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " Jul 28 16:23:18.158428 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.158361 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-trusted-ca\") pod \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " Jul 28 16:23:18.158428 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.158390 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-image-registry-private-configuration\") pod \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " Jul 28 16:23:18.158700 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.158432 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tj87z\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-kube-api-access-tj87z\") pod \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " Jul 28 16:23:18.158700 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.158474 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-ca-trust-extracted\") pod \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\" (UID: \"bae9c53c-7ff2-43b0-b0ac-5433a37338e4\") " Jul 28 16:23:18.158808 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.158734 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bae9c53c-7ff2-43b0-b0ac-5433a37338e4" (UID: "bae9c53c-7ff2-43b0-b0ac-5433a37338e4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:23:18.159077 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.159045 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bae9c53c-7ff2-43b0-b0ac-5433a37338e4" (UID: "bae9c53c-7ff2-43b0-b0ac-5433a37338e4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:23:18.161268 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.161242 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bae9c53c-7ff2-43b0-b0ac-5433a37338e4" (UID: "bae9c53c-7ff2-43b0-b0ac-5433a37338e4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:23:18.161371 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.161238 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bae9c53c-7ff2-43b0-b0ac-5433a37338e4" (UID: "bae9c53c-7ff2-43b0-b0ac-5433a37338e4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:23:18.161371 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.161350 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-kube-api-access-tj87z" (OuterVolumeSpecName: "kube-api-access-tj87z") pod "bae9c53c-7ff2-43b0-b0ac-5433a37338e4" (UID: "bae9c53c-7ff2-43b0-b0ac-5433a37338e4"). InnerVolumeSpecName "kube-api-access-tj87z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:23:18.161449 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.161383 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bae9c53c-7ff2-43b0-b0ac-5433a37338e4" (UID: "bae9c53c-7ff2-43b0-b0ac-5433a37338e4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:23:18.161449 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.161411 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bae9c53c-7ff2-43b0-b0ac-5433a37338e4" (UID: "bae9c53c-7ff2-43b0-b0ac-5433a37338e4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:23:18.167201 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.167180 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bae9c53c-7ff2-43b0-b0ac-5433a37338e4" (UID: "bae9c53c-7ff2-43b0-b0ac-5433a37338e4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 16:23:18.260022 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.259952 2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-ca-trust-extracted\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:23:18.260022 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.260012 2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-installation-pull-secrets\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:23:18.260022 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.260022 2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-bound-sa-token\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:23:18.260022 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.260030 2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-certificates\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:23:18.260289 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.260041 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-registry-tls\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:23:18.260289 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.260051 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-trusted-ca\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:23:18.260289 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.260062 2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-image-registry-private-configuration\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:23:18.260289 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.260070 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj87z\" (UniqueName: \"kubernetes.io/projected/bae9c53c-7ff2-43b0-b0ac-5433a37338e4-kube-api-access-tj87z\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:23:18.872102 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.872065 2566 generic.go:356] "Generic (PLEG): container finished" podID="51fb28cb-f191-418e-ae54-43ca310712ec" containerID="df392f63b90bc2b22cf8d4b1b5f33b8c0cec752f0f8f66a337f98f6e467d373b" exitCode=0 Jul 28 16:23:18.872570 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.872141 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n" event={"ID":"51fb28cb-f191-418e-ae54-43ca310712ec","Type":"ContainerDied","Data":"df392f63b90bc2b22cf8d4b1b5f33b8c0cec752f0f8f66a337f98f6e467d373b"} Jul 28 16:23:18.872570 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.872509 2566 scope.go:117] "RemoveContainer" containerID="df392f63b90bc2b22cf8d4b1b5f33b8c0cec752f0f8f66a337f98f6e467d373b" Jul 28 16:23:18.873533 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.873513 2566 generic.go:356] "Generic (PLEG): container finished" podID="bae9c53c-7ff2-43b0-b0ac-5433a37338e4" containerID="5e2807d83e725e9381a47ce9a140d77dfbd2ac41a77ba2ce2db1d1f6afd5ba95" exitCode=0 Jul 28 16:23:18.873655 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.873556 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-554f97cd94-4klh7" Jul 28 16:23:18.873655 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.873593 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-554f97cd94-4klh7" event={"ID":"bae9c53c-7ff2-43b0-b0ac-5433a37338e4","Type":"ContainerDied","Data":"5e2807d83e725e9381a47ce9a140d77dfbd2ac41a77ba2ce2db1d1f6afd5ba95"} Jul 28 16:23:18.873655 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.873635 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-554f97cd94-4klh7" event={"ID":"bae9c53c-7ff2-43b0-b0ac-5433a37338e4","Type":"ContainerDied","Data":"08df79612230aa572c0c27abd66928867e61308bea8b12058fc0ce3b331184a5"} Jul 28 16:23:18.873655 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.873655 2566 scope.go:117] "RemoveContainer" containerID="5e2807d83e725e9381a47ce9a140d77dfbd2ac41a77ba2ce2db1d1f6afd5ba95" Jul 28 16:23:18.875898 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.875866 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-9hwqx" event={"ID":"676a1e19-6f6c-42a4-881c-730a584c1c2b","Type":"ContainerStarted","Data":"c6ae501bbc8b43841fe650663c778c45d08d5b66a876ba2c564702bf34f2bb99"} Jul 28 16:23:18.887636 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.887607 2566 scope.go:117] "RemoveContainer" containerID="5e2807d83e725e9381a47ce9a140d77dfbd2ac41a77ba2ce2db1d1f6afd5ba95" Jul 28 16:23:18.888052 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:23:18.887966 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e2807d83e725e9381a47ce9a140d77dfbd2ac41a77ba2ce2db1d1f6afd5ba95\": container with ID starting with 5e2807d83e725e9381a47ce9a140d77dfbd2ac41a77ba2ce2db1d1f6afd5ba95 not found: ID does not exist" containerID="5e2807d83e725e9381a47ce9a140d77dfbd2ac41a77ba2ce2db1d1f6afd5ba95" Jul 28 16:23:18.888052 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.888020 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e2807d83e725e9381a47ce9a140d77dfbd2ac41a77ba2ce2db1d1f6afd5ba95"} err="failed to get container status \"5e2807d83e725e9381a47ce9a140d77dfbd2ac41a77ba2ce2db1d1f6afd5ba95\": rpc error: code = NotFound desc = could not find container \"5e2807d83e725e9381a47ce9a140d77dfbd2ac41a77ba2ce2db1d1f6afd5ba95\": container with ID starting with 5e2807d83e725e9381a47ce9a140d77dfbd2ac41a77ba2ce2db1d1f6afd5ba95 not found: ID does not exist" Jul 28 16:23:18.949543 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.949515 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-554f97cd94-4klh7"] Jul 28 16:23:18.967711 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:18.967669 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-554f97cd94-4klh7"] Jul 28 16:23:19.114440 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:19.114407 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bae9c53c-7ff2-43b0-b0ac-5433a37338e4" path="/var/lib/kubelet/pods/bae9c53c-7ff2-43b0-b0ac-5433a37338e4/volumes" Jul 28 16:23:19.882521 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:19.882488 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n" event={"ID":"51fb28cb-f191-418e-ae54-43ca310712ec","Type":"ContainerStarted","Data":"93a601a4df051fdc6306bd73eb6e5942f77588307ba3a627e1540e70df5f8077"} Jul 28 16:23:28.876158 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:28.876128 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:23:28.880507 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:28.880483 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7b9878b988-mgx2k" Jul 28 16:23:50.745297 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:50.745203 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:23:50.761622 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:50.761597 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:23:51.030058 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:23:51.029957 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:04.463120 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:04.463081 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:24:05.070072 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:05.069999 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="alertmanager" containerID="cri-o://0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b" gracePeriod=120 Jul 28 16:24:05.070072 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:05.070034 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="kube-rbac-proxy-web" containerID="cri-o://1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589" gracePeriod=120 Jul 28 16:24:05.070072 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:05.070067 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="prom-label-proxy" containerID="cri-o://becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3" gracePeriod=120 Jul 28 16:24:05.070429 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:05.070065 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="config-reloader" containerID="cri-o://3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b" gracePeriod=120 Jul 28 16:24:05.070429 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:05.070056 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="kube-rbac-proxy" containerID="cri-o://b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576" gracePeriod=120 Jul 28 16:24:05.070429 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:05.070026 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="kube-rbac-proxy-metric" containerID="cri-o://d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0" gracePeriod=120 Jul 28 16:24:06.077100 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.077067 2566 generic.go:356] "Generic (PLEG): container finished" podID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerID="becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3" exitCode=0 Jul 28 16:24:06.077100 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.077094 2566 generic.go:356] "Generic (PLEG): container finished" podID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerID="b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576" exitCode=0 Jul 28 16:24:06.077100 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.077102 2566 generic.go:356] "Generic (PLEG): container finished" podID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerID="3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b" exitCode=0 Jul 28 16:24:06.077100 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.077108 2566 generic.go:356] "Generic (PLEG): container finished" podID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerID="0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b" exitCode=0 Jul 28 16:24:06.077620 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.077139 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1553d0e-49bb-4ee5-8ddd-cfa991b34745","Type":"ContainerDied","Data":"becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3"} Jul 28 16:24:06.077620 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.077224 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1553d0e-49bb-4ee5-8ddd-cfa991b34745","Type":"ContainerDied","Data":"b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576"} Jul 28 16:24:06.077620 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.077268 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1553d0e-49bb-4ee5-8ddd-cfa991b34745","Type":"ContainerDied","Data":"3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b"} Jul 28 16:24:06.077620 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.077308 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1553d0e-49bb-4ee5-8ddd-cfa991b34745","Type":"ContainerDied","Data":"0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b"} Jul 28 16:24:06.325613 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.325591 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:06.495960 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.495899 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-main-tls\") pod \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " Jul 28 16:24:06.496203 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.496037 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-config-volume\") pod \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " Jul 28 16:24:06.496203 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.496063 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-tls-assets\") pod \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " Jul 28 16:24:06.496203 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.496090 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-alertmanager-main-db\") pod \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " Jul 28 16:24:06.496203 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.496122 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy\") pod \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " Jul 28 16:24:06.496203 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.496166 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy-metric\") pod \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " Jul 28 16:24:06.496203 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.496195 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy-web\") pod \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " Jul 28 16:24:06.496504 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.496225 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-cluster-tls-config\") pod \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " Jul 28 16:24:06.496504 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.496445 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "f1553d0e-49bb-4ee5-8ddd-cfa991b34745" (UID: "f1553d0e-49bb-4ee5-8ddd-cfa991b34745"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 16:24:06.496611 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.496547 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwkph\" (UniqueName: \"kubernetes.io/projected/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-kube-api-access-cwkph\") pod \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " Jul 28 16:24:06.496611 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.496592 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-config-out\") pod \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " Jul 28 16:24:06.496702 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.496637 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-alertmanager-trusted-ca-bundle\") pod \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " Jul 28 16:24:06.498894 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.498693 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-metrics-client-ca\") pod \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " Jul 28 16:24:06.498894 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.498759 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-web-config\") pod \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\" (UID: \"f1553d0e-49bb-4ee5-8ddd-cfa991b34745\") " Jul 28 16:24:06.499488 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.499447 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "f1553d0e-49bb-4ee5-8ddd-cfa991b34745" (UID: "f1553d0e-49bb-4ee5-8ddd-cfa991b34745"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:24:06.499573 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.499473 2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-alertmanager-main-db\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:06.500080 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.499943 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "f1553d0e-49bb-4ee5-8ddd-cfa991b34745" (UID: "f1553d0e-49bb-4ee5-8ddd-cfa991b34745"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:24:06.500280 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.500253 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "f1553d0e-49bb-4ee5-8ddd-cfa991b34745" (UID: "f1553d0e-49bb-4ee5-8ddd-cfa991b34745"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:24:06.502020 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.500931 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "f1553d0e-49bb-4ee5-8ddd-cfa991b34745" (UID: "f1553d0e-49bb-4ee5-8ddd-cfa991b34745"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:24:06.502020 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.500958 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "f1553d0e-49bb-4ee5-8ddd-cfa991b34745" (UID: "f1553d0e-49bb-4ee5-8ddd-cfa991b34745"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:24:06.502235 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.502196 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-kube-api-access-cwkph" (OuterVolumeSpecName: "kube-api-access-cwkph") pod "f1553d0e-49bb-4ee5-8ddd-cfa991b34745" (UID: "f1553d0e-49bb-4ee5-8ddd-cfa991b34745"). InnerVolumeSpecName "kube-api-access-cwkph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:24:06.504100 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.504075 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "f1553d0e-49bb-4ee5-8ddd-cfa991b34745" (UID: "f1553d0e-49bb-4ee5-8ddd-cfa991b34745"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:24:06.504224 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.504118 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-config-volume" (OuterVolumeSpecName: "config-volume") pod "f1553d0e-49bb-4ee5-8ddd-cfa991b34745" (UID: "f1553d0e-49bb-4ee5-8ddd-cfa991b34745"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:24:06.504307 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.504134 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f1553d0e-49bb-4ee5-8ddd-cfa991b34745" (UID: "f1553d0e-49bb-4ee5-8ddd-cfa991b34745"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:24:06.504936 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.504909 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-config-out" (OuterVolumeSpecName: "config-out") pod "f1553d0e-49bb-4ee5-8ddd-cfa991b34745" (UID: "f1553d0e-49bb-4ee5-8ddd-cfa991b34745"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 16:24:06.508080 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.508051 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "f1553d0e-49bb-4ee5-8ddd-cfa991b34745" (UID: "f1553d0e-49bb-4ee5-8ddd-cfa991b34745"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:24:06.516936 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.516914 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-web-config" (OuterVolumeSpecName: "web-config") pod "f1553d0e-49bb-4ee5-8ddd-cfa991b34745" (UID: "f1553d0e-49bb-4ee5-8ddd-cfa991b34745"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:24:06.600834 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.600792 2566 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:06.600834 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.600830 2566 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-metrics-client-ca\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:06.600834 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.600841 2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-web-config\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:06.600834 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.600850 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-main-tls\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:06.601087 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.600859 2566 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-config-volume\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:06.601087 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.600866 2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-tls-assets\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:06.601087 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.600874 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:06.601087 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.600883 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:06.601087 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.600892 2566 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:06.601087 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.600900 2566 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-cluster-tls-config\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:06.601087 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.600909 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwkph\" (UniqueName: \"kubernetes.io/projected/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-kube-api-access-cwkph\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:06.601087 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:06.600916 2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1553d0e-49bb-4ee5-8ddd-cfa991b34745-config-out\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:07.084061 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.084026 2566 generic.go:356] "Generic (PLEG): container finished" podID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerID="d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0" exitCode=0 Jul 28 16:24:07.084061 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.084058 2566 generic.go:356] "Generic (PLEG): container finished" podID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerID="1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589" exitCode=0 Jul 28 16:24:07.084569 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.084108 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1553d0e-49bb-4ee5-8ddd-cfa991b34745","Type":"ContainerDied","Data":"d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0"} Jul 28 16:24:07.084569 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.084133 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.084569 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.084237 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1553d0e-49bb-4ee5-8ddd-cfa991b34745","Type":"ContainerDied","Data":"1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589"} Jul 28 16:24:07.084569 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.084282 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f1553d0e-49bb-4ee5-8ddd-cfa991b34745","Type":"ContainerDied","Data":"1f8f93974862c8a945517b01f97558ec9e7b7b0fd983d4ed15e592d0ccfa948c"} Jul 28 16:24:07.084569 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.084306 2566 scope.go:117] "RemoveContainer" containerID="becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3" Jul 28 16:24:07.092263 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.092107 2566 scope.go:117] "RemoveContainer" containerID="d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0" Jul 28 16:24:07.099540 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.099521 2566 scope.go:117] "RemoveContainer" containerID="b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576" Jul 28 16:24:07.107599 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.107582 2566 scope.go:117] "RemoveContainer" containerID="1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589" Jul 28 16:24:07.110583 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.110560 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:24:07.113593 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.113574 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:24:07.116053 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.116035 2566 scope.go:117] "RemoveContainer" containerID="3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b" Jul 28 16:24:07.123025 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.123005 2566 scope.go:117] "RemoveContainer" containerID="0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b" Jul 28 16:24:07.130090 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.130075 2566 scope.go:117] "RemoveContainer" containerID="284cb63b01d65a3f19eae9ed14bff5e32151863e5f879927ef212635bbc7e65d" Jul 28 16:24:07.136843 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.136826 2566 scope.go:117] "RemoveContainer" containerID="becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3" Jul 28 16:24:07.137145 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:24:07.137121 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3\": container with ID starting with becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3 not found: ID does not exist" containerID="becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3" Jul 28 16:24:07.137211 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.137158 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3"} err="failed to get container status \"becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3\": rpc error: code = NotFound desc = could not find container \"becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3\": container with ID starting with becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3 not found: ID does not exist" Jul 28 16:24:07.137211 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.137186 2566 scope.go:117] "RemoveContainer" containerID="d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0" Jul 28 16:24:07.137475 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:24:07.137455 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0\": container with ID starting with d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0 not found: ID does not exist" containerID="d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0" Jul 28 16:24:07.137537 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.137483 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0"} err="failed to get container status \"d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0\": rpc error: code = NotFound desc = could not find container \"d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0\": container with ID starting with d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0 not found: ID does not exist" Jul 28 16:24:07.137537 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.137503 2566 scope.go:117] "RemoveContainer" containerID="b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576" Jul 28 16:24:07.137756 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:24:07.137739 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576\": container with ID starting with b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576 not found: ID does not exist" containerID="b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576" Jul 28 16:24:07.137816 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.137763 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576"} err="failed to get container status \"b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576\": rpc error: code = NotFound desc = could not find container \"b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576\": container with ID starting with b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576 not found: ID does not exist" Jul 28 16:24:07.137816 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.137778 2566 scope.go:117] "RemoveContainer" containerID="1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589" Jul 28 16:24:07.138040 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:24:07.138024 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589\": container with ID starting with 1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589 not found: ID does not exist" containerID="1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589" Jul 28 16:24:07.138101 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.138045 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589"} err="failed to get container status \"1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589\": rpc error: code = NotFound desc = could not find container \"1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589\": container with ID starting with 1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589 not found: ID does not exist" Jul 28 16:24:07.138101 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.138058 2566 scope.go:117] "RemoveContainer" containerID="3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b" Jul 28 16:24:07.138303 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:24:07.138287 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b\": container with ID starting with 3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b not found: ID does not exist" containerID="3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b" Jul 28 16:24:07.138343 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.138308 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b"} err="failed to get container status \"3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b\": rpc error: code = NotFound desc = could not find container \"3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b\": container with ID starting with 3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b not found: ID does not exist" Jul 28 16:24:07.138343 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.138322 2566 scope.go:117] "RemoveContainer" containerID="0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b" Jul 28 16:24:07.138556 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:24:07.138542 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b\": container with ID starting with 0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b not found: ID does not exist" containerID="0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b" Jul 28 16:24:07.138595 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.138559 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b"} err="failed to get container status \"0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b\": rpc error: code = NotFound desc = could not find container \"0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b\": container with ID starting with 0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b not found: ID does not exist" Jul 28 16:24:07.138595 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.138571 2566 scope.go:117] "RemoveContainer" containerID="284cb63b01d65a3f19eae9ed14bff5e32151863e5f879927ef212635bbc7e65d" Jul 28 16:24:07.138805 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:24:07.138789 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"284cb63b01d65a3f19eae9ed14bff5e32151863e5f879927ef212635bbc7e65d\": container with ID starting with 284cb63b01d65a3f19eae9ed14bff5e32151863e5f879927ef212635bbc7e65d not found: ID does not exist" containerID="284cb63b01d65a3f19eae9ed14bff5e32151863e5f879927ef212635bbc7e65d" Jul 28 16:24:07.138853 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.138809 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"284cb63b01d65a3f19eae9ed14bff5e32151863e5f879927ef212635bbc7e65d"} err="failed to get container status \"284cb63b01d65a3f19eae9ed14bff5e32151863e5f879927ef212635bbc7e65d\": rpc error: code = NotFound desc = could not find container \"284cb63b01d65a3f19eae9ed14bff5e32151863e5f879927ef212635bbc7e65d\": container with ID starting with 284cb63b01d65a3f19eae9ed14bff5e32151863e5f879927ef212635bbc7e65d not found: ID does not exist" Jul 28 16:24:07.138853 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.138822 2566 scope.go:117] "RemoveContainer" containerID="becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3" Jul 28 16:24:07.139024 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.139005 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3"} err="failed to get container status \"becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3\": rpc error: code = NotFound desc = could not find container \"becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3\": container with ID starting with becf5f3db88f23191e53cd014b036d5f76c950274500011f62bbacb585d174f3 not found: ID does not exist" Jul 28 16:24:07.139075 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.139025 2566 scope.go:117] "RemoveContainer" containerID="d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0" Jul 28 16:24:07.139237 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.139221 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0"} err="failed to get container status \"d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0\": rpc error: code = NotFound desc = could not find container \"d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0\": container with ID starting with d24aa2f454c92c2627f3e41f86a54cbb38987dcf9a2a6a5e365feef83b10fbe0 not found: ID does not exist" Jul 28 16:24:07.139288 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.139237 2566 scope.go:117] "RemoveContainer" containerID="b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576" Jul 28 16:24:07.139465 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.139450 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576"} err="failed to get container status \"b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576\": rpc error: code = NotFound desc = could not find container \"b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576\": container with ID starting with b59bfa00d085a1ebc44ff42d116a86b12d3ddf9f6ec13b23432b985c19e6e576 not found: ID does not exist" Jul 28 16:24:07.139504 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.139465 2566 scope.go:117] "RemoveContainer" containerID="1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589" Jul 28 16:24:07.139661 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.139645 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589"} err="failed to get container status \"1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589\": rpc error: code = NotFound desc = could not find container \"1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589\": container with ID starting with 1d9bacfd44f236706a1ed0be059c4f71971b6e3e7827967331f0c60b6a9c0589 not found: ID does not exist" Jul 28 16:24:07.139661 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.139660 2566 scope.go:117] "RemoveContainer" containerID="3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b" Jul 28 16:24:07.139844 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.139829 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b"} err="failed to get container status \"3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b\": rpc error: code = NotFound desc = could not find container \"3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b\": container with ID starting with 3666d0c794792fb72531973747ce6fb28aef443bf1a4291114ce9131ac587f3b not found: ID does not exist" Jul 28 16:24:07.139895 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.139844 2566 scope.go:117] "RemoveContainer" containerID="0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b" Jul 28 16:24:07.140082 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.140066 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b"} err="failed to get container status \"0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b\": rpc error: code = NotFound desc = could not find container \"0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b\": container with ID starting with 0a89d22171637885bc29003298ab180703e561cec71d8b5e5457f8505a9d4d7b not found: ID does not exist" Jul 28 16:24:07.140082 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.140081 2566 scope.go:117] "RemoveContainer" containerID="284cb63b01d65a3f19eae9ed14bff5e32151863e5f879927ef212635bbc7e65d" Jul 28 16:24:07.140453 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.140392 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"284cb63b01d65a3f19eae9ed14bff5e32151863e5f879927ef212635bbc7e65d"} err="failed to get container status \"284cb63b01d65a3f19eae9ed14bff5e32151863e5f879927ef212635bbc7e65d\": rpc error: code = NotFound desc = could not find container \"284cb63b01d65a3f19eae9ed14bff5e32151863e5f879927ef212635bbc7e65d\": container with ID starting with 284cb63b01d65a3f19eae9ed14bff5e32151863e5f879927ef212635bbc7e65d not found: ID does not exist" Jul 28 16:24:07.150437 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150416 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:24:07.150723 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150711 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="init-config-reloader" Jul 28 16:24:07.150762 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150725 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="init-config-reloader" Jul 28 16:24:07.150762 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150734 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="alertmanager" Jul 28 16:24:07.150762 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150740 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="alertmanager" Jul 28 16:24:07.150762 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150751 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="config-reloader" Jul 28 16:24:07.150762 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150757 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="config-reloader" Jul 28 16:24:07.150762 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150763 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="kube-rbac-proxy-metric" Jul 28 16:24:07.150943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150770 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="kube-rbac-proxy-metric" Jul 28 16:24:07.150943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150787 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="kube-rbac-proxy" Jul 28 16:24:07.150943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150792 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="kube-rbac-proxy" Jul 28 16:24:07.150943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150804 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="prom-label-proxy" Jul 28 16:24:07.150943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150809 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="prom-label-proxy" Jul 28 16:24:07.150943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150817 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="kube-rbac-proxy-web" Jul 28 16:24:07.150943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150822 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="kube-rbac-proxy-web" Jul 28 16:24:07.150943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150829 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bae9c53c-7ff2-43b0-b0ac-5433a37338e4" containerName="registry" Jul 28 16:24:07.150943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150834 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bae9c53c-7ff2-43b0-b0ac-5433a37338e4" containerName="registry" Jul 28 16:24:07.150943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150888 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="prom-label-proxy" Jul 28 16:24:07.150943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150898 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bae9c53c-7ff2-43b0-b0ac-5433a37338e4" containerName="registry" Jul 28 16:24:07.150943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150903 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="kube-rbac-proxy-metric" Jul 28 16:24:07.150943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150910 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="alertmanager" Jul 28 16:24:07.150943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150914 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="config-reloader" Jul 28 16:24:07.150943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150919 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="kube-rbac-proxy" Jul 28 16:24:07.150943 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.150923 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" containerName="kube-rbac-proxy-web" Jul 28 16:24:07.155506 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.155486 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.159167 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.158954 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 16:24:07.159167 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.159000 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 16:24:07.159167 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.159003 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 16:24:07.159167 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.159022 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fq657\"" Jul 28 16:24:07.159167 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.159049 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 16:24:07.159167 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.159123 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 16:24:07.159546 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.159219 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 16:24:07.159604 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.159561 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 16:24:07.159690 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.159672 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 16:24:07.164721 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.164704 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 16:24:07.176361 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.176338 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:24:07.307151 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.307115 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.307331 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.307157 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-config-volume\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.307331 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.307176 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.307331 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.307240 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.307331 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.307262 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.307331 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.307291 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-web-config\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.307331 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.307312 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wnfs\" (UniqueName: \"kubernetes.io/projected/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-kube-api-access-6wnfs\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.307331 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.307332 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.307584 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.307365 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.307584 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.307394 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.307584 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.307410 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.307584 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.307430 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-config-out\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.307584 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.307457 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.408902 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.408828 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.408902 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.408864 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.408902 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.408895 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-web-config\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.409187 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.408925 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wnfs\" (UniqueName: \"kubernetes.io/projected/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-kube-api-access-6wnfs\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.409187 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.408954 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.409187 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.409011 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.409336 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.409187 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.409336 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.409228 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.409336 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.409262 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-config-out\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.409336 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.409290 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.409336 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.409308 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.409608 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.409363 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.409608 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.409407 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-config-volume\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.409608 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.409434 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.410196 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.410171 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.410545 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.410518 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.412172 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.412037 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.412172 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.412128 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.412308 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.412223 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.412460 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.412439 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.412900 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.412879 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-config-out\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.413048 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.413029 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-web-config\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.413193 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.413176 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.413686 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.413668 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-config-volume\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.414104 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.414083 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.417699 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.417680 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wnfs\" (UniqueName: \"kubernetes.io/projected/f3f67baf-e5be-4222-b890-cc2fc9b3f5f5-kube-api-access-6wnfs\") pod \"alertmanager-main-0\" (UID: \"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.466049 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.466023 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:24:07.613760 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:07.613733 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:24:07.615628 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:24:07.615601 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3f67baf_e5be_4222_b890_cc2fc9b3f5f5.slice/crio-0df6f57820aa3bd13379e680f24156b2231c71356159c03cd1370587c3c2212a WatchSource:0}: Error finding container 0df6f57820aa3bd13379e680f24156b2231c71356159c03cd1370587c3c2212a: Status 404 returned error can't find the container with id 0df6f57820aa3bd13379e680f24156b2231c71356159c03cd1370587c3c2212a Jul 28 16:24:08.089816 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:08.089775 2566 generic.go:356] "Generic (PLEG): container finished" podID="f3f67baf-e5be-4222-b890-cc2fc9b3f5f5" containerID="1554e7b6d1bebd4f0849a3f2a2dea57ce1578480e4fb4ca89368bfbf2c0a9e8f" exitCode=0 Jul 28 16:24:08.090512 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:08.089863 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5","Type":"ContainerDied","Data":"1554e7b6d1bebd4f0849a3f2a2dea57ce1578480e4fb4ca89368bfbf2c0a9e8f"} Jul 28 16:24:08.090512 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:08.089960 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5","Type":"ContainerStarted","Data":"0df6f57820aa3bd13379e680f24156b2231c71356159c03cd1370587c3c2212a"} Jul 28 16:24:08.796956 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:08.796913 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 16:24:08.797414 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:08.797237 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="prometheus" containerID="cri-o://6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9" gracePeriod=600 Jul 28 16:24:08.797414 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:08.797257 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="kube-rbac-proxy" containerID="cri-o://a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff" gracePeriod=600 Jul 28 16:24:08.797414 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:08.797295 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="kube-rbac-proxy-thanos" containerID="cri-o://ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d" gracePeriod=600 Jul 28 16:24:08.797414 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:08.797286 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="kube-rbac-proxy-web" containerID="cri-o://7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003" gracePeriod=600 Jul 28 16:24:08.797414 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:08.797322 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="config-reloader" containerID="cri-o://8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb" gracePeriod=600 Jul 28 16:24:08.797414 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:08.797266 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="thanos-sidecar" containerID="cri-o://7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9" gracePeriod=600 Jul 28 16:24:09.066691 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.066664 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.108993 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.108895 2566 generic.go:356] "Generic (PLEG): container finished" podID="fd11b80b-529d-4497-bafb-0293f59966f6" containerID="ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d" exitCode=0 Jul 28 16:24:09.108993 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.108927 2566 generic.go:356] "Generic (PLEG): container finished" podID="fd11b80b-529d-4497-bafb-0293f59966f6" containerID="a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff" exitCode=0 Jul 28 16:24:09.108993 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.108937 2566 generic.go:356] "Generic (PLEG): container finished" podID="fd11b80b-529d-4497-bafb-0293f59966f6" containerID="7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003" exitCode=0 Jul 28 16:24:09.108993 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.108945 2566 generic.go:356] "Generic (PLEG): container finished" podID="fd11b80b-529d-4497-bafb-0293f59966f6" containerID="7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9" exitCode=0 Jul 28 16:24:09.108993 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.108952 2566 generic.go:356] "Generic (PLEG): container finished" podID="fd11b80b-529d-4497-bafb-0293f59966f6" containerID="8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb" exitCode=0 Jul 28 16:24:09.108993 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.108960 2566 generic.go:356] "Generic (PLEG): container finished" podID="fd11b80b-529d-4497-bafb-0293f59966f6" containerID="6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9" exitCode=0 Jul 28 16:24:09.109598 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.109182 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.114020 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.112237 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1553d0e-49bb-4ee5-8ddd-cfa991b34745" path="/var/lib/kubelet/pods/f1553d0e-49bb-4ee5-8ddd-cfa991b34745/volumes" Jul 28 16:24:09.115252 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.115212 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd11b80b-529d-4497-bafb-0293f59966f6","Type":"ContainerDied","Data":"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d"} Jul 28 16:24:09.115395 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.115296 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd11b80b-529d-4497-bafb-0293f59966f6","Type":"ContainerDied","Data":"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff"} Jul 28 16:24:09.115395 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.115347 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd11b80b-529d-4497-bafb-0293f59966f6","Type":"ContainerDied","Data":"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003"} Jul 28 16:24:09.115395 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.115382 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd11b80b-529d-4497-bafb-0293f59966f6","Type":"ContainerDied","Data":"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9"} Jul 28 16:24:09.115552 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.115425 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd11b80b-529d-4497-bafb-0293f59966f6","Type":"ContainerDied","Data":"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb"} Jul 28 16:24:09.115552 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.115380 2566 scope.go:117] "RemoveContainer" containerID="ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d" Jul 28 16:24:09.115552 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.115468 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd11b80b-529d-4497-bafb-0293f59966f6","Type":"ContainerDied","Data":"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9"} Jul 28 16:24:09.115552 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.115510 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fd11b80b-529d-4497-bafb-0293f59966f6","Type":"ContainerDied","Data":"ace4c84a397e60f5551f6a6844232cf52db2ffb41e3c288051c008ad646f1a1a"} Jul 28 16:24:09.115964 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.115872 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5","Type":"ContainerStarted","Data":"82629b6fd5b52b28475143f264986aeebb6637e99b63b595f043add012f6980b"} Jul 28 16:24:09.115964 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.115953 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5","Type":"ContainerStarted","Data":"402704b611a23f1c5634b3ac90f4e1b9415345c30ea69e0be0b9dd78c858bf49"} Jul 28 16:24:09.116124 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.116023 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5","Type":"ContainerStarted","Data":"b1e39c4620f2d7554a28024b85040915497088d91c621812a09e273320d609ae"} Jul 28 16:24:09.116124 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.116065 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5","Type":"ContainerStarted","Data":"28919a3d5ff8a08b16e0c5cc2a0db9ddd79751d7ec7d4a8fa913430cbd7d8d27"} Jul 28 16:24:09.116124 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.116105 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5","Type":"ContainerStarted","Data":"ca12c9baad17fe9de60a3085ac33e89c9d82eaf1ab7303869197a6252c7859e6"} Jul 28 16:24:09.116302 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.116144 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f3f67baf-e5be-4222-b890-cc2fc9b3f5f5","Type":"ContainerStarted","Data":"8db33447f96f0a3f29594de88f6fc2125be61355e450f77db0e42aa0281116f8"} Jul 28 16:24:09.126889 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.126865 2566 scope.go:117] "RemoveContainer" containerID="a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff" Jul 28 16:24:09.136605 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.136511 2566 scope.go:117] "RemoveContainer" containerID="7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003" Jul 28 16:24:09.146558 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.146520 2566 scope.go:117] "RemoveContainer" containerID="7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9" Jul 28 16:24:09.163834 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.163792 2566 scope.go:117] "RemoveContainer" containerID="8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb" Jul 28 16:24:09.171786 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.171769 2566 scope.go:117] "RemoveContainer" containerID="6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9" Jul 28 16:24:09.179782 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.179759 2566 scope.go:117] "RemoveContainer" containerID="54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052" Jul 28 16:24:09.190000 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.189965 2566 scope.go:117] "RemoveContainer" containerID="ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d" Jul 28 16:24:09.190305 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:24:09.190284 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d\": container with ID starting with ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d not found: ID does not exist" containerID="ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d" Jul 28 16:24:09.190387 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.190320 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d"} err="failed to get container status \"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d\": rpc error: code = NotFound desc = could not find container \"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d\": container with ID starting with ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d not found: ID does not exist" Jul 28 16:24:09.190387 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.190346 2566 scope.go:117] "RemoveContainer" containerID="a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff" Jul 28 16:24:09.190656 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:24:09.190626 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff\": container with ID starting with a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff not found: ID does not exist" containerID="a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff" Jul 28 16:24:09.190705 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.190662 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff"} err="failed to get container status \"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff\": rpc error: code = NotFound desc = could not find container \"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff\": container with ID starting with a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff not found: ID does not exist" Jul 28 16:24:09.190705 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.190678 2566 scope.go:117] "RemoveContainer" containerID="7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003" Jul 28 16:24:09.190953 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:24:09.190931 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003\": container with ID starting with 7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003 not found: ID does not exist" containerID="7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003" Jul 28 16:24:09.191096 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.191078 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003"} err="failed to get container status \"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003\": rpc error: code = NotFound desc = could not find container \"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003\": container with ID starting with 7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003 not found: ID does not exist" Jul 28 16:24:09.191139 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.191100 2566 scope.go:117] "RemoveContainer" containerID="7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9" Jul 28 16:24:09.191373 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:24:09.191354 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9\": container with ID starting with 7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9 not found: ID does not exist" containerID="7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9" Jul 28 16:24:09.191431 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.191377 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9"} err="failed to get container status \"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9\": rpc error: code = NotFound desc = could not find container \"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9\": container with ID starting with 7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9 not found: ID does not exist" Jul 28 16:24:09.191431 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.191397 2566 scope.go:117] "RemoveContainer" containerID="8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb" Jul 28 16:24:09.191632 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:24:09.191614 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb\": container with ID starting with 8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb not found: ID does not exist" containerID="8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb" Jul 28 16:24:09.191694 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.191635 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb"} err="failed to get container status \"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb\": rpc error: code = NotFound desc = could not find container \"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb\": container with ID starting with 8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb not found: ID does not exist" Jul 28 16:24:09.191694 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.191647 2566 scope.go:117] "RemoveContainer" containerID="6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9" Jul 28 16:24:09.191871 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:24:09.191855 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9\": container with ID starting with 6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9 not found: ID does not exist" containerID="6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9" Jul 28 16:24:09.191937 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.191874 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9"} err="failed to get container status \"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9\": rpc error: code = NotFound desc = could not find container \"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9\": container with ID starting with 6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9 not found: ID does not exist" Jul 28 16:24:09.191937 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.191888 2566 scope.go:117] "RemoveContainer" containerID="54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052" Jul 28 16:24:09.192144 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:24:09.192126 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052\": container with ID starting with 54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052 not found: ID does not exist" containerID="54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052" Jul 28 16:24:09.192181 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.192149 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052"} err="failed to get container status \"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052\": rpc error: code = NotFound desc = could not find container \"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052\": container with ID starting with 54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052 not found: ID does not exist" Jul 28 16:24:09.192181 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.192165 2566 scope.go:117] "RemoveContainer" containerID="ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d" Jul 28 16:24:09.192407 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.192391 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d"} err="failed to get container status \"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d\": rpc error: code = NotFound desc = could not find container \"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d\": container with ID starting with ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d not found: ID does not exist" Jul 28 16:24:09.192447 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.192408 2566 scope.go:117] "RemoveContainer" containerID="a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff" Jul 28 16:24:09.192603 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.192587 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff"} err="failed to get container status \"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff\": rpc error: code = NotFound desc = could not find container \"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff\": container with ID starting with a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff not found: ID does not exist" Jul 28 16:24:09.192642 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.192605 2566 scope.go:117] "RemoveContainer" containerID="7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003" Jul 28 16:24:09.192803 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.192785 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003"} err="failed to get container status \"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003\": rpc error: code = NotFound desc = could not find container \"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003\": container with ID starting with 7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003 not found: ID does not exist" Jul 28 16:24:09.192845 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.192804 2566 scope.go:117] "RemoveContainer" containerID="7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9" Jul 28 16:24:09.193016 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.192995 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9"} err="failed to get container status \"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9\": rpc error: code = NotFound desc = could not find container \"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9\": container with ID starting with 7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9 not found: ID does not exist" Jul 28 16:24:09.193079 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.193017 2566 scope.go:117] "RemoveContainer" containerID="8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb" Jul 28 16:24:09.193237 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.193221 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb"} err="failed to get container status \"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb\": rpc error: code = NotFound desc = could not find container \"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb\": container with ID starting with 8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb not found: ID does not exist" Jul 28 16:24:09.193278 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.193238 2566 scope.go:117] "RemoveContainer" containerID="6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9" Jul 28 16:24:09.193450 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.193434 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9"} err="failed to get container status \"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9\": rpc error: code = NotFound desc = could not find container \"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9\": container with ID starting with 6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9 not found: ID does not exist" Jul 28 16:24:09.193488 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.193451 2566 scope.go:117] "RemoveContainer" containerID="54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052" Jul 28 16:24:09.193636 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.193619 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052"} err="failed to get container status \"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052\": rpc error: code = NotFound desc = could not find container \"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052\": container with ID starting with 54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052 not found: ID does not exist" Jul 28 16:24:09.193677 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.193636 2566 scope.go:117] "RemoveContainer" containerID="ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d" Jul 28 16:24:09.193825 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.193810 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d"} err="failed to get container status \"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d\": rpc error: code = NotFound desc = could not find container \"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d\": container with ID starting with ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d not found: ID does not exist" Jul 28 16:24:09.193873 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.193826 2566 scope.go:117] "RemoveContainer" containerID="a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff" Jul 28 16:24:09.194055 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.194040 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff"} err="failed to get container status \"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff\": rpc error: code = NotFound desc = could not find container \"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff\": container with ID starting with a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff not found: ID does not exist" Jul 28 16:24:09.194107 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.194056 2566 scope.go:117] "RemoveContainer" containerID="7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003" Jul 28 16:24:09.194231 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.194216 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003"} err="failed to get container status \"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003\": rpc error: code = NotFound desc = could not find container \"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003\": container with ID starting with 7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003 not found: ID does not exist" Jul 28 16:24:09.194269 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.194232 2566 scope.go:117] "RemoveContainer" containerID="7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9" Jul 28 16:24:09.194412 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.194397 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9"} err="failed to get container status \"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9\": rpc error: code = NotFound desc = could not find container \"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9\": container with ID starting with 7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9 not found: ID does not exist" Jul 28 16:24:09.194457 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.194412 2566 scope.go:117] "RemoveContainer" containerID="8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb" Jul 28 16:24:09.194579 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.194565 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb"} err="failed to get container status \"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb\": rpc error: code = NotFound desc = could not find container \"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb\": container with ID starting with 8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb not found: ID does not exist" Jul 28 16:24:09.194621 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.194580 2566 scope.go:117] "RemoveContainer" containerID="6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9" Jul 28 16:24:09.194749 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.194733 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9"} err="failed to get container status \"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9\": rpc error: code = NotFound desc = could not find container \"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9\": container with ID starting with 6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9 not found: ID does not exist" Jul 28 16:24:09.194804 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.194749 2566 scope.go:117] "RemoveContainer" containerID="54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052" Jul 28 16:24:09.194957 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.194936 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052"} err="failed to get container status \"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052\": rpc error: code = NotFound desc = could not find container \"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052\": container with ID starting with 54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052 not found: ID does not exist" Jul 28 16:24:09.194957 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.194957 2566 scope.go:117] "RemoveContainer" containerID="ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d" Jul 28 16:24:09.195227 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.195209 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d"} err="failed to get container status \"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d\": rpc error: code = NotFound desc = could not find container \"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d\": container with ID starting with ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d not found: ID does not exist" Jul 28 16:24:09.195275 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.195229 2566 scope.go:117] "RemoveContainer" containerID="a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff" Jul 28 16:24:09.195440 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.195421 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff"} err="failed to get container status \"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff\": rpc error: code = NotFound desc = could not find container \"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff\": container with ID starting with a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff not found: ID does not exist" Jul 28 16:24:09.195440 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.195440 2566 scope.go:117] "RemoveContainer" containerID="7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003" Jul 28 16:24:09.195630 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.195612 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003"} err="failed to get container status \"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003\": rpc error: code = NotFound desc = could not find container \"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003\": container with ID starting with 7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003 not found: ID does not exist" Jul 28 16:24:09.195675 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.195631 2566 scope.go:117] "RemoveContainer" containerID="7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9" Jul 28 16:24:09.195844 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.195827 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9"} err="failed to get container status \"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9\": rpc error: code = NotFound desc = could not find container \"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9\": container with ID starting with 7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9 not found: ID does not exist" Jul 28 16:24:09.195905 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.195845 2566 scope.go:117] "RemoveContainer" containerID="8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb" Jul 28 16:24:09.196076 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.196060 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb"} err="failed to get container status \"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb\": rpc error: code = NotFound desc = could not find container \"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb\": container with ID starting with 8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb not found: ID does not exist" Jul 28 16:24:09.196130 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.196077 2566 scope.go:117] "RemoveContainer" containerID="6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9" Jul 28 16:24:09.196278 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.196260 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9"} err="failed to get container status \"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9\": rpc error: code = NotFound desc = could not find container \"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9\": container with ID starting with 6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9 not found: ID does not exist" Jul 28 16:24:09.196323 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.196278 2566 scope.go:117] "RemoveContainer" containerID="54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052" Jul 28 16:24:09.196457 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.196440 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052"} err="failed to get container status \"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052\": rpc error: code = NotFound desc = could not find container \"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052\": container with ID starting with 54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052 not found: ID does not exist" Jul 28 16:24:09.196503 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.196458 2566 scope.go:117] "RemoveContainer" containerID="ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d" Jul 28 16:24:09.196666 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.196650 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d"} err="failed to get container status \"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d\": rpc error: code = NotFound desc = could not find container \"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d\": container with ID starting with ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d not found: ID does not exist" Jul 28 16:24:09.196712 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.196666 2566 scope.go:117] "RemoveContainer" containerID="a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff" Jul 28 16:24:09.196870 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.196854 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff"} err="failed to get container status \"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff\": rpc error: code = NotFound desc = could not find container \"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff\": container with ID starting with a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff not found: ID does not exist" Jul 28 16:24:09.196870 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.196870 2566 scope.go:117] "RemoveContainer" containerID="7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003" Jul 28 16:24:09.197091 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.197074 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003"} err="failed to get container status \"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003\": rpc error: code = NotFound desc = could not find container \"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003\": container with ID starting with 7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003 not found: ID does not exist" Jul 28 16:24:09.197146 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.197091 2566 scope.go:117] "RemoveContainer" containerID="7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9" Jul 28 16:24:09.197289 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.197274 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9"} err="failed to get container status \"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9\": rpc error: code = NotFound desc = could not find container \"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9\": container with ID starting with 7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9 not found: ID does not exist" Jul 28 16:24:09.197334 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.197290 2566 scope.go:117] "RemoveContainer" containerID="8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb" Jul 28 16:24:09.197475 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.197454 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb"} err="failed to get container status \"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb\": rpc error: code = NotFound desc = could not find container \"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb\": container with ID starting with 8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb not found: ID does not exist" Jul 28 16:24:09.197519 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.197477 2566 scope.go:117] "RemoveContainer" containerID="6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9" Jul 28 16:24:09.197677 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.197660 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9"} err="failed to get container status \"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9\": rpc error: code = NotFound desc = could not find container \"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9\": container with ID starting with 6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9 not found: ID does not exist" Jul 28 16:24:09.197719 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.197678 2566 scope.go:117] "RemoveContainer" containerID="54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052" Jul 28 16:24:09.197875 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.197860 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052"} err="failed to get container status \"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052\": rpc error: code = NotFound desc = could not find container \"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052\": container with ID starting with 54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052 not found: ID does not exist" Jul 28 16:24:09.197930 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.197875 2566 scope.go:117] "RemoveContainer" containerID="ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d" Jul 28 16:24:09.198083 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.198068 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d"} err="failed to get container status \"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d\": rpc error: code = NotFound desc = could not find container \"ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d\": container with ID starting with ee1dc595cbd9e8e152589b1651172c092f63a43f6703793f9abcfd2ec6bbe28d not found: ID does not exist" Jul 28 16:24:09.198083 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.198083 2566 scope.go:117] "RemoveContainer" containerID="a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff" Jul 28 16:24:09.198292 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.198274 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff"} err="failed to get container status \"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff\": rpc error: code = NotFound desc = could not find container \"a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff\": container with ID starting with a38242df35f441a27daed6277b45c3c11c9efb35f333416dd79a70abe0c21cff not found: ID does not exist" Jul 28 16:24:09.198340 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.198291 2566 scope.go:117] "RemoveContainer" containerID="7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003" Jul 28 16:24:09.198507 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.198492 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003"} err="failed to get container status \"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003\": rpc error: code = NotFound desc = could not find container \"7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003\": container with ID starting with 7e41d079563ab2d1f7fa6a8c954fe3ffa717bb708f334f351f1dbf07d641d003 not found: ID does not exist" Jul 28 16:24:09.198551 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.198508 2566 scope.go:117] "RemoveContainer" containerID="7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9" Jul 28 16:24:09.198703 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.198684 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9"} err="failed to get container status \"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9\": rpc error: code = NotFound desc = could not find container \"7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9\": container with ID starting with 7ab94569b31b7455dd2768e303214783c2a240135553fd8fd86da2cac11297a9 not found: ID does not exist" Jul 28 16:24:09.198755 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.198704 2566 scope.go:117] "RemoveContainer" containerID="8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb" Jul 28 16:24:09.198893 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.198878 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb"} err="failed to get container status \"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb\": rpc error: code = NotFound desc = could not find container \"8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb\": container with ID starting with 8f5fab33acf8328de61561ac25c56eae8911c2e2f511c91de68d556c136e82cb not found: ID does not exist" Jul 28 16:24:09.198940 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.198895 2566 scope.go:117] "RemoveContainer" containerID="6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9" Jul 28 16:24:09.199120 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.199102 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9"} err="failed to get container status \"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9\": rpc error: code = NotFound desc = could not find container \"6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9\": container with ID starting with 6c05b5fb482b99cb6ce245eb45f92530b0be29703fc88d4a07cc8bcc0ec92bc9 not found: ID does not exist" Jul 28 16:24:09.199178 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.199120 2566 scope.go:117] "RemoveContainer" containerID="54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052" Jul 28 16:24:09.199318 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.199303 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052"} err="failed to get container status \"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052\": rpc error: code = NotFound desc = could not find container \"54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052\": container with ID starting with 54a082f5db2563921ff9e6f9a3475401a593b1c89426e16063f0be44cf077052 not found: ID does not exist" Jul 28 16:24:09.225654 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.225632 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fd11b80b-529d-4497-bafb-0293f59966f6-tls-assets\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.225781 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.225680 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.225781 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.225705 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.225931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.225860 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-config\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.225931 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.225908 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-db\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.226174 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.225939 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fd11b80b-529d-4497-bafb-0293f59966f6-config-out\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.226174 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.225967 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-trusted-ca-bundle\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.226174 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.226027 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-metrics-client-certs\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.226174 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.226059 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-kube-rbac-proxy\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.226174 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.226093 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t82xj\" (UniqueName: \"kubernetes.io/projected/fd11b80b-529d-4497-bafb-0293f59966f6-kube-api-access-t82xj\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.226174 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.226152 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-grpc-tls\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.226835 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.226181 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-0\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.226835 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.226218 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-kubelet-serving-ca-bundle\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.226835 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.226248 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-tls\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.226835 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.226272 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-web-config\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.226835 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.226296 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-metrics-client-ca\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.226835 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.226442 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:24:09.226835 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.226482 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-2\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.226835 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.226517 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-thanos-prometheus-http-client-file\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.226835 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.226543 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-1\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.226835 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.226580 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-serving-certs-ca-bundle\") pod \"fd11b80b-529d-4497-bafb-0293f59966f6\" (UID: \"fd11b80b-529d-4497-bafb-0293f59966f6\") " Jul 28 16:24:09.227639 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.227136 2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.227639 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.227390 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:24:09.228489 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.228458 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 16:24:09.228723 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.228698 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:24:09.229795 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.229766 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:24:09.230112 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.229996 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:24:09.230443 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.230417 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:24:09.230523 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.230476 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:24:09.230708 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.230678 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd11b80b-529d-4497-bafb-0293f59966f6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:24:09.230708 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.230693 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:24:09.230862 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.230794 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:24:09.231213 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.231133 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd11b80b-529d-4497-bafb-0293f59966f6-kube-api-access-t82xj" (OuterVolumeSpecName: "kube-api-access-t82xj") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "kube-api-access-t82xj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:24:09.231213 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.231149 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:24:09.231213 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.231207 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:24:09.231365 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.231231 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-config" (OuterVolumeSpecName: "config") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:24:09.231489 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.231468 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:24:09.232061 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.232030 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd11b80b-529d-4497-bafb-0293f59966f6-config-out" (OuterVolumeSpecName: "config-out") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 16:24:09.232522 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.232500 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:24:09.232571 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.232522 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:24:09.245624 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.245547 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-web-config" (OuterVolumeSpecName: "web-config") pod "fd11b80b-529d-4497-bafb-0293f59966f6" (UID: "fd11b80b-529d-4497-bafb-0293f59966f6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:24:09.328080 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328004 2566 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-kube-rbac-proxy\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328080 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328032 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t82xj\" (UniqueName: \"kubernetes.io/projected/fd11b80b-529d-4497-bafb-0293f59966f6-kube-api-access-t82xj\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328080 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328047 2566 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-grpc-tls\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328080 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328059 2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328080 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328071 2566 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328359 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328086 2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328359 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328100 2566 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-web-config\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328359 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328111 2566 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-metrics-client-ca\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328359 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328122 2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328359 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328134 2566 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328359 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328146 2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328359 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328157 2566 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd11b80b-529d-4497-bafb-0293f59966f6-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328359 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328169 2566 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fd11b80b-529d-4497-bafb-0293f59966f6-tls-assets\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328359 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328182 2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328359 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328195 2566 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328359 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328206 2566 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-config\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328359 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328218 2566 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fd11b80b-529d-4497-bafb-0293f59966f6-prometheus-k8s-db\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328359 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328230 2566 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fd11b80b-529d-4497-bafb-0293f59966f6-config-out\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.328359 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.328242 2566 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fd11b80b-529d-4497-bafb-0293f59966f6-secret-metrics-client-certs\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:24:09.433815 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.433761 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.417877378 podStartE2EDuration="2.433741844s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.864466ms" isStatefulPod=false podCreationTimestamp="2026-07-28 16:24:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:24:09.146545199 +0000 UTC m=+190.528734853" watchObservedRunningTime="2026-07-28 16:24:09.433741844 +0000 UTC m=+190.815931456" Jul 28 16:24:09.434808 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.434790 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 16:24:09.439329 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.439305 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 16:24:09.475600 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.475570 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 16:24:09.476041 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476022 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="kube-rbac-proxy-web" Jul 28 16:24:09.476102 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476045 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="kube-rbac-proxy-web" Jul 28 16:24:09.476102 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476076 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="kube-rbac-proxy-thanos" Jul 28 16:24:09.476102 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476085 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="kube-rbac-proxy-thanos" Jul 28 16:24:09.476194 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476100 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="config-reloader" Jul 28 16:24:09.476194 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476111 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="config-reloader" Jul 28 16:24:09.476194 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476121 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="init-config-reloader" Jul 28 16:24:09.476194 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476129 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="init-config-reloader" Jul 28 16:24:09.476194 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476141 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="prometheus" Jul 28 16:24:09.476194 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476148 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="prometheus" Jul 28 16:24:09.476194 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476158 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="kube-rbac-proxy" Jul 28 16:24:09.476194 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476165 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="kube-rbac-proxy" Jul 28 16:24:09.476194 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476181 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="thanos-sidecar" Jul 28 16:24:09.476194 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476190 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="thanos-sidecar" Jul 28 16:24:09.476519 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476284 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="thanos-sidecar" Jul 28 16:24:09.476519 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476299 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="prometheus" Jul 28 16:24:09.476519 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476307 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="config-reloader" Jul 28 16:24:09.476519 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476315 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="kube-rbac-proxy-web" Jul 28 16:24:09.476519 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476324 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="kube-rbac-proxy" Jul 28 16:24:09.476519 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.476335 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" containerName="kube-rbac-proxy-thanos" Jul 28 16:24:09.479902 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.479883 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.482683 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.482661 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 28 16:24:09.482787 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.482702 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 28 16:24:09.482787 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.482702 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 28 16:24:09.482918 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.482823 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 28 16:24:09.483113 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.483094 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 28 16:24:09.483236 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.483150 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 28 16:24:09.483236 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.483199 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 28 16:24:09.483516 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.483499 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 28 16:24:09.483516 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.483453 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 28 16:24:09.483718 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.483699 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rvgww\"" Jul 28 16:24:09.483832 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.483811 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 28 16:24:09.483930 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.483699 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 28 16:24:09.483999 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.483722 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-m51utjhoeb6t\"" Jul 28 16:24:09.484363 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.484152 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 28 16:24:09.495124 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.489792 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 28 16:24:09.496339 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.496315 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 28 16:24:09.500443 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.500415 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 16:24:09.630621 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.630532 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-web-config\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.630621 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.630574 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-config\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.630621 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.630592 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.630862 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.630657 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.630862 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.630705 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.630862 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.630731 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.630862 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.630754 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.630862 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.630773 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.630862 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.630814 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.630862 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.630858 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.631106 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.630877 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-config-out\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.631106 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.630896 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.631106 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.630912 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvlff\" (UniqueName: \"kubernetes.io/projected/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-kube-api-access-vvlff\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.631106 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.630931 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.631106 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.631000 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.631106 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.631029 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.631106 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.631049 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.631300 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.631111 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.631300 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.631128 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.631300 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.631151 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.731879 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.731832 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.732021 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.731896 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.732021 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.731938 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.732021 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.731965 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.732021 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.732017 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.732230 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.732042 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.732230 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.732080 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.732230 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.732106 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-config-out\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.732230 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.732131 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.732230 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.732159 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvlff\" (UniqueName: \"kubernetes.io/projected/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-kube-api-access-vvlff\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.732450 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.732425 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.732761 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.732736 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.733002 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.732952 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.733296 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.733278 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.733902 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.733378 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.733902 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.733392 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.733902 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.733450 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.733902 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.733501 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.733902 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.733530 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.733902 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.733569 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.733902 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.733575 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.733902 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.733594 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-web-config\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.733902 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.733645 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-config\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.733902 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.733676 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.734456 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.734396 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.735414 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.735385 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-config-out\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.735414 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.735582 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.735414 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.735713 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.735414 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.735780 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.735414 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.736047 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.737495 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.736719 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.737495 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.737119 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-web-config\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.737495 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.737437 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.737653 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.737621 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.737705 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.737689 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-config\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.737819 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.737799 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.738201 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.738179 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.738316 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.738225 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.739123 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.739104 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.740800 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.740778 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvlff\" (UniqueName: \"kubernetes.io/projected/252febaf-ea3e-4cc0-be7f-d287f1a2f9a9-kube-api-access-vvlff\") pod \"prometheus-k8s-0\" (UID: \"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.794478 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.794437 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:24:09.936035 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:09.936016 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 16:24:09.938317 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:24:09.938295 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod252febaf_ea3e_4cc0_be7f_d287f1a2f9a9.slice/crio-5294763ed4a4359f93a88ae085ff18a974fdbc6f8857d83ebcf36ee1a86dcdc8 WatchSource:0}: Error finding container 5294763ed4a4359f93a88ae085ff18a974fdbc6f8857d83ebcf36ee1a86dcdc8: Status 404 returned error can't find the container with id 5294763ed4a4359f93a88ae085ff18a974fdbc6f8857d83ebcf36ee1a86dcdc8 Jul 28 16:24:10.121922 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:10.121884 2566 generic.go:356] "Generic (PLEG): container finished" podID="252febaf-ea3e-4cc0-be7f-d287f1a2f9a9" containerID="07b66ccb8e72b28ba767a8e4e75b7247afff49b8d8787d0826ec14902ab4e7f4" exitCode=0 Jul 28 16:24:10.122319 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:10.121994 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9","Type":"ContainerDied","Data":"07b66ccb8e72b28ba767a8e4e75b7247afff49b8d8787d0826ec14902ab4e7f4"} Jul 28 16:24:10.122319 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:10.122063 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9","Type":"ContainerStarted","Data":"5294763ed4a4359f93a88ae085ff18a974fdbc6f8857d83ebcf36ee1a86dcdc8"} Jul 28 16:24:11.107881 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:11.107846 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd11b80b-529d-4497-bafb-0293f59966f6" path="/var/lib/kubelet/pods/fd11b80b-529d-4497-bafb-0293f59966f6/volumes" Jul 28 16:24:11.129366 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:11.129332 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9","Type":"ContainerStarted","Data":"78cfd23330df152cb7b29b92234ecbbf2faa98ce7348c104d84e841d7a9353a2"} Jul 28 16:24:11.129745 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:11.129403 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9","Type":"ContainerStarted","Data":"57a1d2e646a5a1e5efaa1c90258ccdb2d1003b7d1e91938ff17d4a73b24509b7"} Jul 28 16:24:11.129745 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:11.129432 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9","Type":"ContainerStarted","Data":"2126bb97a0169bb5c2bf2a0449ebfd4324fa3045d0729a38b5ad6027ffdb18d4"} Jul 28 16:24:11.129745 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:11.129461 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9","Type":"ContainerStarted","Data":"6e16b5256c9753e66c846fc7a099d6adc95fa86288c476912fc8507e306ab144"} Jul 28 16:24:11.129745 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:11.129489 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9","Type":"ContainerStarted","Data":"08f0489b51a401460abbb162fbb78db588c4782c1e4a95e0e438390c7baeb797"} Jul 28 16:24:11.129745 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:11.129513 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"252febaf-ea3e-4cc0-be7f-d287f1a2f9a9","Type":"ContainerStarted","Data":"14d681f676a45a4006e519941a6f091b93555d6aaaf838cc6ae414323738aaec"} Jul 28 16:24:11.158291 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:11.158237 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.141910999 podStartE2EDuration="2.158211664s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.300665ms" isStatefulPod=false podCreationTimestamp="2026-07-28 16:24:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:24:11.155777279 +0000 UTC m=+192.537966925" watchObservedRunningTime="2026-07-28 16:24:11.158211664 +0000 UTC m=+192.540401282" Jul 28 16:24:14.795072 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:24:14.795034 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:25:09.795059 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:25:09.795022 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:25:09.812570 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:25:09.812544 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:25:10.385381 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:25:10.385357 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 16:25:58.972187 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:25:58.972157 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:25:58.976020 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:25:58.975994 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:25:58.978379 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:25:58.978355 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:25:58.982528 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:25:58.982508 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:25:58.983899 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:25:58.983880 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 28 16:30:32.471826 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:32.471784 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-clpcc"] Jul 28 16:30:32.475109 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:32.475094 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-clpcc" Jul 28 16:30:32.477538 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:32.477514 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-vcflw\"" Jul 28 16:30:32.477773 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:32.477758 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 28 16:30:32.478848 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:32.478833 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 28 16:30:32.478935 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:32.478907 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 28 16:30:32.481916 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:32.481887 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-clpcc"] Jul 28 16:30:32.614849 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:32.614810 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffb26a63-5ce3-40e8-991b-b997e95c6e0f-model-cache\") pod \"s3-init-clpcc\" (UID: \"ffb26a63-5ce3-40e8-991b-b997e95c6e0f\") " pod="kserve/s3-init-clpcc" Jul 28 16:30:32.615069 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:32.614887 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rwm7\" (UniqueName: \"kubernetes.io/projected/ffb26a63-5ce3-40e8-991b-b997e95c6e0f-kube-api-access-2rwm7\") pod \"s3-init-clpcc\" (UID: \"ffb26a63-5ce3-40e8-991b-b997e95c6e0f\") " pod="kserve/s3-init-clpcc" Jul 28 16:30:32.716386 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:32.716345 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rwm7\" (UniqueName: \"kubernetes.io/projected/ffb26a63-5ce3-40e8-991b-b997e95c6e0f-kube-api-access-2rwm7\") pod \"s3-init-clpcc\" (UID: \"ffb26a63-5ce3-40e8-991b-b997e95c6e0f\") " pod="kserve/s3-init-clpcc" Jul 28 16:30:32.716559 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:32.716395 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffb26a63-5ce3-40e8-991b-b997e95c6e0f-model-cache\") pod \"s3-init-clpcc\" (UID: \"ffb26a63-5ce3-40e8-991b-b997e95c6e0f\") " pod="kserve/s3-init-clpcc" Jul 28 16:30:32.716736 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:32.716719 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffb26a63-5ce3-40e8-991b-b997e95c6e0f-model-cache\") pod \"s3-init-clpcc\" (UID: \"ffb26a63-5ce3-40e8-991b-b997e95c6e0f\") " pod="kserve/s3-init-clpcc" Jul 28 16:30:32.724662 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:32.724606 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rwm7\" (UniqueName: \"kubernetes.io/projected/ffb26a63-5ce3-40e8-991b-b997e95c6e0f-kube-api-access-2rwm7\") pod \"s3-init-clpcc\" (UID: \"ffb26a63-5ce3-40e8-991b-b997e95c6e0f\") " pod="kserve/s3-init-clpcc" Jul 28 16:30:32.797685 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:32.797654 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-clpcc" Jul 28 16:30:32.932000 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:32.931958 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-clpcc"] Jul 28 16:30:32.933994 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:30:32.933953 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podffb26a63_5ce3_40e8_991b_b997e95c6e0f.slice/crio-e8c6dbff289486227da8a05383b437cc3a64e79f316a84b85987b5b0b216300a WatchSource:0}: Error finding container e8c6dbff289486227da8a05383b437cc3a64e79f316a84b85987b5b0b216300a: Status 404 returned error can't find the container with id e8c6dbff289486227da8a05383b437cc3a64e79f316a84b85987b5b0b216300a Jul 28 16:30:32.941737 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:32.936431 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 16:30:33.645444 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:33.645399 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-clpcc" event={"ID":"ffb26a63-5ce3-40e8-991b-b997e95c6e0f","Type":"ContainerStarted","Data":"e8c6dbff289486227da8a05383b437cc3a64e79f316a84b85987b5b0b216300a"} Jul 28 16:30:37.667906 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:37.667868 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-clpcc" event={"ID":"ffb26a63-5ce3-40e8-991b-b997e95c6e0f","Type":"ContainerStarted","Data":"6e9b1578ccd7e30cdab7caabcb2afe8ba2579d8421f2842677843dab768975bc"} Jul 28 16:30:42.690686 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:42.690652 2566 generic.go:356] "Generic (PLEG): container finished" podID="ffb26a63-5ce3-40e8-991b-b997e95c6e0f" containerID="6e9b1578ccd7e30cdab7caabcb2afe8ba2579d8421f2842677843dab768975bc" exitCode=0 Jul 28 16:30:42.691091 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:42.690733 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-clpcc" event={"ID":"ffb26a63-5ce3-40e8-991b-b997e95c6e0f","Type":"ContainerDied","Data":"6e9b1578ccd7e30cdab7caabcb2afe8ba2579d8421f2842677843dab768975bc"} Jul 28 16:30:47.717395 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:47.717361 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-clpcc" event={"ID":"ffb26a63-5ce3-40e8-991b-b997e95c6e0f","Type":"ContainerStarted","Data":"5da29c72280cf9401472b3c361032462347bffd9924cbb5f8c8440f6c9322b6c"} Jul 28 16:30:47.734296 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:47.734245 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-clpcc" podStartSLOduration=2.348397747 podStartE2EDuration="15.734227825s" totalImagesPullingTime="8.161208583s" totalInitContainerRuntime="5.224621495s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:30:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:30:47.732772447 +0000 UTC m=+589.114962066" watchObservedRunningTime="2026-07-28 16:30:47.734227825 +0000 UTC m=+589.116417444" Jul 28 16:30:59.008109 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:59.008080 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:30:59.008645 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:59.008080 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:30:59.013746 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:59.013726 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:30:59.013895 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:30:59.013790 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:31:52.974833 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:31:52.974798 2566 generic.go:356] "Generic (PLEG): container finished" podID="ffb26a63-5ce3-40e8-991b-b997e95c6e0f" containerID="5da29c72280cf9401472b3c361032462347bffd9924cbb5f8c8440f6c9322b6c" exitCode=0 Jul 28 16:31:52.975350 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:31:52.974872 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-clpcc" event={"ID":"ffb26a63-5ce3-40e8-991b-b997e95c6e0f","Type":"ContainerDied","Data":"5da29c72280cf9401472b3c361032462347bffd9924cbb5f8c8440f6c9322b6c"} Jul 28 16:31:54.110283 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:31:54.110261 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-clpcc" Jul 28 16:31:54.227871 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:31:54.227833 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rwm7\" (UniqueName: \"kubernetes.io/projected/ffb26a63-5ce3-40e8-991b-b997e95c6e0f-kube-api-access-2rwm7\") pod \"ffb26a63-5ce3-40e8-991b-b997e95c6e0f\" (UID: \"ffb26a63-5ce3-40e8-991b-b997e95c6e0f\") " Jul 28 16:31:54.228093 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:31:54.227899 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffb26a63-5ce3-40e8-991b-b997e95c6e0f-model-cache\") pod \"ffb26a63-5ce3-40e8-991b-b997e95c6e0f\" (UID: \"ffb26a63-5ce3-40e8-991b-b997e95c6e0f\") " Jul 28 16:31:54.230314 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:31:54.230255 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffb26a63-5ce3-40e8-991b-b997e95c6e0f-kube-api-access-2rwm7" (OuterVolumeSpecName: "kube-api-access-2rwm7") pod "ffb26a63-5ce3-40e8-991b-b997e95c6e0f" (UID: "ffb26a63-5ce3-40e8-991b-b997e95c6e0f"). InnerVolumeSpecName "kube-api-access-2rwm7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:31:54.248009 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:31:54.247959 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb26a63-5ce3-40e8-991b-b997e95c6e0f-model-cache" (OuterVolumeSpecName: "model-cache") pod "ffb26a63-5ce3-40e8-991b-b997e95c6e0f" (UID: "ffb26a63-5ce3-40e8-991b-b997e95c6e0f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 16:31:54.328833 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:31:54.328796 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rwm7\" (UniqueName: \"kubernetes.io/projected/ffb26a63-5ce3-40e8-991b-b997e95c6e0f-kube-api-access-2rwm7\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:31:54.328833 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:31:54.328828 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffb26a63-5ce3-40e8-991b-b997e95c6e0f-model-cache\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:31:54.984268 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:31:54.984238 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-clpcc" Jul 28 16:31:54.984439 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:31:54.984245 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-clpcc" event={"ID":"ffb26a63-5ce3-40e8-991b-b997e95c6e0f","Type":"ContainerDied","Data":"e8c6dbff289486227da8a05383b437cc3a64e79f316a84b85987b5b0b216300a"} Jul 28 16:31:54.984439 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:31:54.984354 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8c6dbff289486227da8a05383b437cc3a64e79f316a84b85987b5b0b216300a" Jul 28 16:32:06.975081 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:06.975043 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986"] Jul 28 16:32:06.975461 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:06.975398 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb26a63-5ce3-40e8-991b-b997e95c6e0f" containerName="download-hf-model" Jul 28 16:32:06.975461 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:06.975410 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb26a63-5ce3-40e8-991b-b997e95c6e0f" containerName="download-hf-model" Jul 28 16:32:06.975461 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:06.975418 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffb26a63-5ce3-40e8-991b-b997e95c6e0f" containerName="s3-init" Jul 28 16:32:06.975461 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:06.975423 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffb26a63-5ce3-40e8-991b-b997e95c6e0f" containerName="s3-init" Jul 28 16:32:06.975597 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:06.975501 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffb26a63-5ce3-40e8-991b-b997e95c6e0f" containerName="s3-init" Jul 28 16:32:06.978981 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:06.978954 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:06.981536 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:06.981511 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\"" Jul 28 16:32:06.981678 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:06.981578 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 28 16:32:06.981802 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:06.981781 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\"" Jul 28 16:32:06.981866 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:06.981809 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 16:32:06.981923 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:06.981879 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 28 16:32:06.982004 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:06.981933 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 28 16:32:06.982846 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:06.982832 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-zwq7z\"" Jul 28 16:32:06.993724 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:06.993697 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986"] Jul 28 16:32:07.039007 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.038956 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgzc4\" (UniqueName: \"kubernetes.io/projected/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kube-api-access-dgzc4\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-2n986\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:07.039179 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.039028 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-2n986\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:07.039179 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.039073 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-2n986\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:07.039179 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.039091 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-2n986\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:07.039179 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.039142 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-2n986\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:07.139745 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.139714 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgzc4\" (UniqueName: \"kubernetes.io/projected/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kube-api-access-dgzc4\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-2n986\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:07.139940 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.139765 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-2n986\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:07.139940 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:32:07.139873 2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-serving-cert: secret "isvc-sklearn-graph-1-predictor-serving-cert" not found Jul 28 16:32:07.139940 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.139897 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-2n986\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:07.139940 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:32:07.139919 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-proxy-tls podName:ba8b1a86-7d9a-476f-89ec-f4b4d99786a6 nodeName:}" failed. No retries permitted until 2026-07-28 16:32:07.639903066 +0000 UTC m=+669.022092661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-proxy-tls") pod "isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" (UID: "ba8b1a86-7d9a-476f-89ec-f4b4d99786a6") : secret "isvc-sklearn-graph-1-predictor-serving-cert" not found Jul 28 16:32:07.139940 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.139938 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-2n986\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:07.140238 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.139964 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-2n986\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:07.140401 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.140380 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-2n986\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:07.140540 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.140524 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-2n986\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:07.140587 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.140573 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-2n986\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:07.148577 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.148554 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgzc4\" (UniqueName: \"kubernetes.io/projected/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kube-api-access-dgzc4\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-2n986\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:07.176767 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.176733 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm"] Jul 28 16:32:07.180339 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.180322 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.182606 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.182587 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\"" Jul 28 16:32:07.182775 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.182759 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\"" Jul 28 16:32:07.191002 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.190961 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm"] Jul 28 16:32:07.240463 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.240429 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-lf2gm\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.240621 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.240476 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3f71a3fc-9d45-415f-8c36-cf045092d0d9-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-lf2gm\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.240621 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.240534 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f71a3fc-9d45-415f-8c36-cf045092d0d9-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-lf2gm\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.240621 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.240595 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-lf2gm\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.240761 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.240632 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z95g\" (UniqueName: \"kubernetes.io/projected/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kube-api-access-8z95g\") pod \"isvc-xgboost-graph-predictor-555fc69d56-lf2gm\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.341371 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.341336 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-lf2gm\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.341371 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.341387 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z95g\" (UniqueName: \"kubernetes.io/projected/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kube-api-access-8z95g\") pod \"isvc-xgboost-graph-predictor-555fc69d56-lf2gm\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.341642 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.341435 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-lf2gm\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.341642 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.341458 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3f71a3fc-9d45-415f-8c36-cf045092d0d9-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-lf2gm\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.341642 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.341619 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f71a3fc-9d45-415f-8c36-cf045092d0d9-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-lf2gm\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.341893 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.341876 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-lf2gm\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.342116 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.342092 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-lf2gm\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.342228 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.342151 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3f71a3fc-9d45-415f-8c36-cf045092d0d9-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-lf2gm\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.344156 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.344138 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f71a3fc-9d45-415f-8c36-cf045092d0d9-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-lf2gm\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.349282 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.349265 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z95g\" (UniqueName: \"kubernetes.io/projected/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kube-api-access-8z95g\") pod \"isvc-xgboost-graph-predictor-555fc69d56-lf2gm\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.491566 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.491478 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:07.626147 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.626115 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm"] Jul 28 16:32:07.629434 ip-10-0-140-22 kubenswrapper[2566]: W0728 16:32:07.629410 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f71a3fc_9d45_415f_8c36_cf045092d0d9.slice/crio-3536dff7bbbd8e312b7c8f7f70f4ffe160d085c6a7eeafe6d57cd02ea1db8c06 WatchSource:0}: Error finding container 3536dff7bbbd8e312b7c8f7f70f4ffe160d085c6a7eeafe6d57cd02ea1db8c06: Status 404 returned error can't find the container with id 3536dff7bbbd8e312b7c8f7f70f4ffe160d085c6a7eeafe6d57cd02ea1db8c06 Jul 28 16:32:07.645420 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.645393 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-2n986\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:07.648217 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.648192 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-2n986\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:07.895590 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:07.895504 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:08.036484 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:08.036454 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986"] Jul 28 16:32:08.046158 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:08.046127 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" event={"ID":"3f71a3fc-9d45-415f-8c36-cf045092d0d9","Type":"ContainerStarted","Data":"bb8dd9797d53ed9d5ea7f28c538c0026ffeebc503aaf996148803c93ff0e2481"} Jul 28 16:32:08.046256 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:08.046187 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" event={"ID":"3f71a3fc-9d45-415f-8c36-cf045092d0d9","Type":"ContainerStarted","Data":"3536dff7bbbd8e312b7c8f7f70f4ffe160d085c6a7eeafe6d57cd02ea1db8c06"} Jul 28 16:32:09.052285 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:09.052243 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" event={"ID":"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6","Type":"ContainerStarted","Data":"eaf101d33fc1fdd6f0dcd4340050cace42e286d02fd7da9f2878da5906f7072c"} Jul 28 16:32:09.052704 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:09.052314 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" event={"ID":"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6","Type":"ContainerStarted","Data":"bde25d25aa017475dc79660d70e0efb2e11bcb9054309f7bf08328c7c7b9bf73"} Jul 28 16:32:12.066020 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:12.065964 2566 generic.go:356] "Generic (PLEG): container finished" podID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerID="eaf101d33fc1fdd6f0dcd4340050cace42e286d02fd7da9f2878da5906f7072c" exitCode=0 Jul 28 16:32:12.066474 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:12.066035 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" event={"ID":"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6","Type":"ContainerDied","Data":"eaf101d33fc1fdd6f0dcd4340050cace42e286d02fd7da9f2878da5906f7072c"} Jul 28 16:32:12.067704 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:12.067678 2566 generic.go:356] "Generic (PLEG): container finished" podID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerID="bb8dd9797d53ed9d5ea7f28c538c0026ffeebc503aaf996148803c93ff0e2481" exitCode=0 Jul 28 16:32:12.067857 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:12.067726 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" event={"ID":"3f71a3fc-9d45-415f-8c36-cf045092d0d9","Type":"ContainerDied","Data":"bb8dd9797d53ed9d5ea7f28c538c0026ffeebc503aaf996148803c93ff0e2481"} Jul 28 16:32:37.214962 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:37.214897 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" event={"ID":"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6","Type":"ContainerStarted","Data":"e2c6fa5002c03877edd72481fdf8a2ac94f210fa0739af72d388efdb549a9c21"} Jul 28 16:32:37.218991 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:37.218906 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" event={"ID":"3f71a3fc-9d45-415f-8c36-cf045092d0d9","Type":"ContainerStarted","Data":"c11ae0a383f803335cbaafec2021af733111bf269347d83727333b32fb924973"} Jul 28 16:32:40.233412 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:40.233368 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" event={"ID":"3f71a3fc-9d45-415f-8c36-cf045092d0d9","Type":"ContainerStarted","Data":"7234d60d22888eef7191cd272893d6e00929a3c1d782bd944b59a184fb99af80"} Jul 28 16:32:40.233900 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:40.233449 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:40.233900 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:40.233482 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:40.234907 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:40.234867 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 28 16:32:40.235741 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:40.235717 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" event={"ID":"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6","Type":"ContainerStarted","Data":"2a08c316cd2476bca14a6d6d9404c67c083a2aeb405736882984212ae15b5928"} Jul 28 16:32:40.235886 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:40.235871 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:40.253838 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:40.253796 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" podStartSLOduration=2.446892672 podStartE2EDuration="33.253780059s" totalImagesPullingTime="27.306830953s" totalInitContainerRuntime="3.500056434s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:32:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:32:40.252118403 +0000 UTC m=+701.634308047" watchObservedRunningTime="2026-07-28 16:32:40.253780059 +0000 UTC m=+701.635969676" Jul 28 16:32:40.270122 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:40.270064 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" podStartSLOduration=3.132993803 podStartE2EDuration="34.270044967s" totalImagesPullingTime="27.320116565s" totalInitContainerRuntime="3.816934599s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:32:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:32:40.269328029 +0000 UTC m=+701.651517644" watchObservedRunningTime="2026-07-28 16:32:40.270044967 +0000 UTC m=+701.652234587" Jul 28 16:32:41.240612 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:41.240574 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:41.241049 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:41.240600 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 28 16:32:41.241807 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:41.241783 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 28 16:32:42.244812 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:42.244771 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 28 16:32:46.245963 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:46.245933 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:32:46.246488 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:46.246463 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 28 16:32:47.249318 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:47.249290 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:32:47.249743 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:47.249718 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 28 16:32:56.247306 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:56.247264 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 28 16:32:57.250125 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:32:57.250087 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 28 16:33:06.246618 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:33:06.246578 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 28 16:33:07.249999 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:33:07.249946 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 28 16:33:16.246830 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:33:16.246790 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 28 16:33:17.249745 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:33:17.249703 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 28 16:33:26.246611 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:33:26.246568 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 28 16:33:27.249718 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:33:27.249683 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 28 16:33:36.246456 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:33:36.246417 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.27:8080: connect: connection refused" Jul 28 16:33:37.249748 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:33:37.249706 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 28 16:33:46.247137 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:33:46.247101 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:33:47.250652 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:33:47.250605 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 28 16:33:57.250136 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:33:57.250104 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:34:37.013708 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:37.013614 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986"] Jul 28 16:34:37.014219 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:37.013920 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kserve-container" containerID="cri-o://e2c6fa5002c03877edd72481fdf8a2ac94f210fa0739af72d388efdb549a9c21" gracePeriod=30 Jul 28 16:34:37.014219 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:37.013944 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kube-rbac-proxy" containerID="cri-o://2a08c316cd2476bca14a6d6d9404c67c083a2aeb405736882984212ae15b5928" gracePeriod=30 Jul 28 16:34:37.075605 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:37.075573 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm"] Jul 28 16:34:37.075865 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:37.075815 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="kserve-container" containerID="cri-o://c11ae0a383f803335cbaafec2021af733111bf269347d83727333b32fb924973" gracePeriod=30 Jul 28 16:34:37.075936 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:37.075874 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="kube-rbac-proxy" containerID="cri-o://7234d60d22888eef7191cd272893d6e00929a3c1d782bd944b59a184fb99af80" gracePeriod=30 Jul 28 16:34:37.245653 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:37.245615 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.26:8643/healthz\": dial tcp 10.132.0.26:8643: connect: connection refused" Jul 28 16:34:37.250073 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:37.250044 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused" Jul 28 16:34:37.737356 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:37.737320 2566 generic.go:356] "Generic (PLEG): container finished" podID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerID="7234d60d22888eef7191cd272893d6e00929a3c1d782bd944b59a184fb99af80" exitCode=2 Jul 28 16:34:37.737542 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:37.737391 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" event={"ID":"3f71a3fc-9d45-415f-8c36-cf045092d0d9","Type":"ContainerDied","Data":"7234d60d22888eef7191cd272893d6e00929a3c1d782bd944b59a184fb99af80"} Jul 28 16:34:37.739296 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:37.739272 2566 generic.go:356] "Generic (PLEG): container finished" podID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerID="2a08c316cd2476bca14a6d6d9404c67c083a2aeb405736882984212ae15b5928" exitCode=2 Jul 28 16:34:37.739417 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:37.739314 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" event={"ID":"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6","Type":"ContainerDied","Data":"2a08c316cd2476bca14a6d6d9404c67c083a2aeb405736882984212ae15b5928"} Jul 28 16:34:40.758445 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:40.758394 2566 generic.go:356] "Generic (PLEG): container finished" podID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerID="c11ae0a383f803335cbaafec2021af733111bf269347d83727333b32fb924973" exitCode=0 Jul 28 16:34:40.758897 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:40.758504 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" event={"ID":"3f71a3fc-9d45-415f-8c36-cf045092d0d9","Type":"ContainerDied","Data":"c11ae0a383f803335cbaafec2021af733111bf269347d83727333b32fb924973"} Jul 28 16:34:40.819110 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:40.819085 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:34:40.904900 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:40.904811 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kube-rbac-proxy-sar-config\") pod \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " Jul 28 16:34:40.904900 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:40.904871 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kserve-provision-location\") pod \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " Jul 28 16:34:40.905153 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:40.904951 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3f71a3fc-9d45-415f-8c36-cf045092d0d9-cabundle-cert\") pod \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " Jul 28 16:34:40.905153 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:40.905002 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f71a3fc-9d45-415f-8c36-cf045092d0d9-proxy-tls\") pod \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " Jul 28 16:34:40.905153 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:40.905045 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8z95g\" (UniqueName: \"kubernetes.io/projected/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kube-api-access-8z95g\") pod \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\" (UID: \"3f71a3fc-9d45-415f-8c36-cf045092d0d9\") " Jul 28 16:34:40.905325 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:40.905283 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "3f71a3fc-9d45-415f-8c36-cf045092d0d9" (UID: "3f71a3fc-9d45-415f-8c36-cf045092d0d9"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 16:34:40.905325 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:40.905292 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3f71a3fc-9d45-415f-8c36-cf045092d0d9" (UID: "3f71a3fc-9d45-415f-8c36-cf045092d0d9"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:34:40.905325 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:40.905311 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f71a3fc-9d45-415f-8c36-cf045092d0d9-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "3f71a3fc-9d45-415f-8c36-cf045092d0d9" (UID: "3f71a3fc-9d45-415f-8c36-cf045092d0d9"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:34:40.907373 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:40.907346 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kube-api-access-8z95g" (OuterVolumeSpecName: "kube-api-access-8z95g") pod "3f71a3fc-9d45-415f-8c36-cf045092d0d9" (UID: "3f71a3fc-9d45-415f-8c36-cf045092d0d9"). InnerVolumeSpecName "kube-api-access-8z95g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:34:40.907373 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:40.907364 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f71a3fc-9d45-415f-8c36-cf045092d0d9-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3f71a3fc-9d45-415f-8c36-cf045092d0d9" (UID: "3f71a3fc-9d45-415f-8c36-cf045092d0d9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:34:41.006538 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:41.006488 2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3f71a3fc-9d45-415f-8c36-cf045092d0d9-cabundle-cert\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:34:41.006538 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:41.006535 2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3f71a3fc-9d45-415f-8c36-cf045092d0d9-proxy-tls\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:34:41.006538 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:41.006544 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8z95g\" (UniqueName: \"kubernetes.io/projected/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kube-api-access-8z95g\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:34:41.006538 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:41.006555 2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:34:41.006822 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:41.006565 2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3f71a3fc-9d45-415f-8c36-cf045092d0d9-kserve-provision-location\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:34:41.764666 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:41.764632 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" event={"ID":"3f71a3fc-9d45-415f-8c36-cf045092d0d9","Type":"ContainerDied","Data":"3536dff7bbbd8e312b7c8f7f70f4ffe160d085c6a7eeafe6d57cd02ea1db8c06"} Jul 28 16:34:41.765128 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:41.764687 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm" Jul 28 16:34:41.765128 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:41.764725 2566 scope.go:117] "RemoveContainer" containerID="7234d60d22888eef7191cd272893d6e00929a3c1d782bd944b59a184fb99af80" Jul 28 16:34:41.773206 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:41.773187 2566 scope.go:117] "RemoveContainer" containerID="c11ae0a383f803335cbaafec2021af733111bf269347d83727333b32fb924973" Jul 28 16:34:41.780738 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:41.780718 2566 scope.go:117] "RemoveContainer" containerID="bb8dd9797d53ed9d5ea7f28c538c0026ffeebc503aaf996148803c93ff0e2481" Jul 28 16:34:41.783840 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:41.783802 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm"] Jul 28 16:34:41.787304 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:41.787278 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-lf2gm"] Jul 28 16:34:42.167208 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.167183 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:34:42.317928 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.317834 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-proxy-tls\") pod \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " Jul 28 16:34:42.317928 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.317874 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-cabundle-cert\") pod \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " Jul 28 16:34:42.317928 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.317927 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kube-rbac-proxy-sar-config\") pod \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " Jul 28 16:34:42.318223 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.317951 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kserve-provision-location\") pod \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " Jul 28 16:34:42.318223 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.318024 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgzc4\" (UniqueName: \"kubernetes.io/projected/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kube-api-access-dgzc4\") pod \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\" (UID: \"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6\") " Jul 28 16:34:42.318340 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.318312 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" (UID: "ba8b1a86-7d9a-476f-89ec-f4b4d99786a6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 16:34:42.318340 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.318331 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" (UID: "ba8b1a86-7d9a-476f-89ec-f4b4d99786a6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:34:42.318408 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.318325 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" (UID: "ba8b1a86-7d9a-476f-89ec-f4b4d99786a6"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:34:42.320097 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.320076 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" (UID: "ba8b1a86-7d9a-476f-89ec-f4b4d99786a6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:34:42.320312 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.320295 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kube-api-access-dgzc4" (OuterVolumeSpecName: "kube-api-access-dgzc4") pod "ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" (UID: "ba8b1a86-7d9a-476f-89ec-f4b4d99786a6"). InnerVolumeSpecName "kube-api-access-dgzc4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:34:42.419498 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.419468 2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-proxy-tls\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:34:42.419498 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.419496 2566 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-cabundle-cert\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:34:42.419679 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.419506 2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:34:42.419679 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.419515 2566 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kserve-provision-location\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:34:42.419679 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.419525 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgzc4\" (UniqueName: \"kubernetes.io/projected/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6-kube-api-access-dgzc4\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 16:34:42.770520 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.770485 2566 generic.go:356] "Generic (PLEG): container finished" podID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerID="e2c6fa5002c03877edd72481fdf8a2ac94f210fa0739af72d388efdb549a9c21" exitCode=0 Jul 28 16:34:42.770950 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.770536 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" Jul 28 16:34:42.770950 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.770570 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" event={"ID":"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6","Type":"ContainerDied","Data":"e2c6fa5002c03877edd72481fdf8a2ac94f210fa0739af72d388efdb549a9c21"} Jul 28 16:34:42.770950 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.770630 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986" event={"ID":"ba8b1a86-7d9a-476f-89ec-f4b4d99786a6","Type":"ContainerDied","Data":"bde25d25aa017475dc79660d70e0efb2e11bcb9054309f7bf08328c7c7b9bf73"} Jul 28 16:34:42.770950 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.770672 2566 scope.go:117] "RemoveContainer" containerID="2a08c316cd2476bca14a6d6d9404c67c083a2aeb405736882984212ae15b5928" Jul 28 16:34:42.779235 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.779219 2566 scope.go:117] "RemoveContainer" containerID="e2c6fa5002c03877edd72481fdf8a2ac94f210fa0739af72d388efdb549a9c21" Jul 28 16:34:42.786661 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.786640 2566 scope.go:117] "RemoveContainer" containerID="eaf101d33fc1fdd6f0dcd4340050cace42e286d02fd7da9f2878da5906f7072c" Jul 28 16:34:42.793061 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.793041 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986"] Jul 28 16:34:42.794667 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.794651 2566 scope.go:117] "RemoveContainer" containerID="2a08c316cd2476bca14a6d6d9404c67c083a2aeb405736882984212ae15b5928" Jul 28 16:34:42.794968 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:34:42.794949 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a08c316cd2476bca14a6d6d9404c67c083a2aeb405736882984212ae15b5928\": container with ID starting with 2a08c316cd2476bca14a6d6d9404c67c083a2aeb405736882984212ae15b5928 not found: ID does not exist" containerID="2a08c316cd2476bca14a6d6d9404c67c083a2aeb405736882984212ae15b5928" Jul 28 16:34:42.795042 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.794994 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a08c316cd2476bca14a6d6d9404c67c083a2aeb405736882984212ae15b5928"} err="failed to get container status \"2a08c316cd2476bca14a6d6d9404c67c083a2aeb405736882984212ae15b5928\": rpc error: code = NotFound desc = could not find container \"2a08c316cd2476bca14a6d6d9404c67c083a2aeb405736882984212ae15b5928\": container with ID starting with 2a08c316cd2476bca14a6d6d9404c67c083a2aeb405736882984212ae15b5928 not found: ID does not exist" Jul 28 16:34:42.795042 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.795013 2566 scope.go:117] "RemoveContainer" containerID="e2c6fa5002c03877edd72481fdf8a2ac94f210fa0739af72d388efdb549a9c21" Jul 28 16:34:42.795263 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:34:42.795245 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2c6fa5002c03877edd72481fdf8a2ac94f210fa0739af72d388efdb549a9c21\": container with ID starting with e2c6fa5002c03877edd72481fdf8a2ac94f210fa0739af72d388efdb549a9c21 not found: ID does not exist" containerID="e2c6fa5002c03877edd72481fdf8a2ac94f210fa0739af72d388efdb549a9c21" Jul 28 16:34:42.795310 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.795270 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2c6fa5002c03877edd72481fdf8a2ac94f210fa0739af72d388efdb549a9c21"} err="failed to get container status \"e2c6fa5002c03877edd72481fdf8a2ac94f210fa0739af72d388efdb549a9c21\": rpc error: code = NotFound desc = could not find container \"e2c6fa5002c03877edd72481fdf8a2ac94f210fa0739af72d388efdb549a9c21\": container with ID starting with e2c6fa5002c03877edd72481fdf8a2ac94f210fa0739af72d388efdb549a9c21 not found: ID does not exist" Jul 28 16:34:42.795310 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.795285 2566 scope.go:117] "RemoveContainer" containerID="eaf101d33fc1fdd6f0dcd4340050cace42e286d02fd7da9f2878da5906f7072c" Jul 28 16:34:42.795523 ip-10-0-140-22 kubenswrapper[2566]: E0728 16:34:42.795502 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaf101d33fc1fdd6f0dcd4340050cace42e286d02fd7da9f2878da5906f7072c\": container with ID starting with eaf101d33fc1fdd6f0dcd4340050cace42e286d02fd7da9f2878da5906f7072c not found: ID does not exist" containerID="eaf101d33fc1fdd6f0dcd4340050cace42e286d02fd7da9f2878da5906f7072c" Jul 28 16:34:42.795578 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.795533 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaf101d33fc1fdd6f0dcd4340050cace42e286d02fd7da9f2878da5906f7072c"} err="failed to get container status \"eaf101d33fc1fdd6f0dcd4340050cace42e286d02fd7da9f2878da5906f7072c\": rpc error: code = NotFound desc = could not find container \"eaf101d33fc1fdd6f0dcd4340050cace42e286d02fd7da9f2878da5906f7072c\": container with ID starting with eaf101d33fc1fdd6f0dcd4340050cace42e286d02fd7da9f2878da5906f7072c not found: ID does not exist" Jul 28 16:34:42.796271 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:42.796254 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-2n986"] Jul 28 16:34:43.107320 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:43.107244 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" path="/var/lib/kubelet/pods/3f71a3fc-9d45-415f-8c36-cf045092d0d9/volumes" Jul 28 16:34:43.107725 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:34:43.107712 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" path="/var/lib/kubelet/pods/ba8b1a86-7d9a-476f-89ec-f4b4d99786a6/volumes" Jul 28 16:35:59.034435 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:35:59.034402 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:35:59.036223 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:35:59.036197 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:35:59.040661 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:35:59.040641 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:35:59.042154 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:35:59.042139 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:40:59.061153 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:40:59.061124 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:40:59.066062 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:40:59.066037 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:40:59.067226 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:40:59.067205 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:40:59.071822 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:40:59.071804 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:45:59.087933 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:45:59.087906 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:45:59.093894 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:45:59.093870 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:45:59.094194 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:45:59.094173 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:45:59.099854 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:45:59.099833 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:50:59.116638 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:50:59.116518 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:50:59.122690 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:50:59.122662 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:50:59.122853 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:50:59.122792 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:50:59.128157 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:50:59.128138 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:55:59.143321 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:55:59.143207 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:55:59.149409 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:55:59.149386 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 16:55:59.152718 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:55:59.152698 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 16:55:59.158092 ip-10-0-140-22 kubenswrapper[2566]: I0728 16:55:59.158072 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 17:00:59.171524 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:00:59.171412 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 17:00:59.177240 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:00:59.177221 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 17:00:59.180366 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:00:59.180347 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 17:00:59.185698 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:00:59.185683 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 17:05:59.198072 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:05:59.197952 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 17:05:59.204081 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:05:59.204051 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 17:05:59.207878 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:05:59.207858 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 17:05:59.213512 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:05:59.213494 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 17:10:59.225373 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:10:59.225270 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 17:10:59.230866 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:10:59.230836 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 17:10:59.235886 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:10:59.235866 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 17:10:59.241989 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:10:59.241955 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 17:11:37.500419 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.500377 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rw7zh/must-gather-x5jx2"] Jul 28 17:11:37.501109 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.500823 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="storage-initializer" Jul 28 17:11:37.501109 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.500841 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="storage-initializer" Jul 28 17:11:37.501109 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.500857 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="kube-rbac-proxy" Jul 28 17:11:37.501109 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.500866 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="kube-rbac-proxy" Jul 28 17:11:37.501109 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.500880 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="storage-initializer" Jul 28 17:11:37.501109 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.500888 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="storage-initializer" Jul 28 17:11:37.501109 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.500899 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kserve-container" Jul 28 17:11:37.501109 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.500907 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kserve-container" Jul 28 17:11:37.501109 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.500918 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kube-rbac-proxy" Jul 28 17:11:37.501109 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.500927 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kube-rbac-proxy" Jul 28 17:11:37.501109 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.500942 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="kserve-container" Jul 28 17:11:37.501109 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.500950 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="kserve-container" Jul 28 17:11:37.501109 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.501074 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="kserve-container" Jul 28 17:11:37.501109 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.501089 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kserve-container" Jul 28 17:11:37.501109 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.501099 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f71a3fc-9d45-415f-8c36-cf045092d0d9" containerName="kube-rbac-proxy" Jul 28 17:11:37.501109 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.501110 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba8b1a86-7d9a-476f-89ec-f4b4d99786a6" containerName="kube-rbac-proxy" Jul 28 17:11:37.504495 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.504474 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rw7zh/must-gather-x5jx2" Jul 28 17:11:37.508062 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.508037 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rw7zh\"/\"kube-root-ca.crt\"" Jul 28 17:11:37.508243 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.508226 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rw7zh\"/\"default-dockercfg-rx4wb\"" Jul 28 17:11:37.508673 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.508660 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rw7zh\"/\"openshift-service-ca.crt\"" Jul 28 17:11:37.517009 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.516961 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rw7zh/must-gather-x5jx2"] Jul 28 17:11:37.569465 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.569424 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/359826bf-3f3a-4acc-8061-1ad4ad5dd832-must-gather-output\") pod \"must-gather-x5jx2\" (UID: \"359826bf-3f3a-4acc-8061-1ad4ad5dd832\") " pod="openshift-must-gather-rw7zh/must-gather-x5jx2" Jul 28 17:11:37.569642 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.569491 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgqcv\" (UniqueName: \"kubernetes.io/projected/359826bf-3f3a-4acc-8061-1ad4ad5dd832-kube-api-access-wgqcv\") pod \"must-gather-x5jx2\" (UID: \"359826bf-3f3a-4acc-8061-1ad4ad5dd832\") " pod="openshift-must-gather-rw7zh/must-gather-x5jx2" Jul 28 17:11:37.670205 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.670147 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgqcv\" (UniqueName: \"kubernetes.io/projected/359826bf-3f3a-4acc-8061-1ad4ad5dd832-kube-api-access-wgqcv\") pod \"must-gather-x5jx2\" (UID: \"359826bf-3f3a-4acc-8061-1ad4ad5dd832\") " pod="openshift-must-gather-rw7zh/must-gather-x5jx2" Jul 28 17:11:37.670401 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.670259 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/359826bf-3f3a-4acc-8061-1ad4ad5dd832-must-gather-output\") pod \"must-gather-x5jx2\" (UID: \"359826bf-3f3a-4acc-8061-1ad4ad5dd832\") " pod="openshift-must-gather-rw7zh/must-gather-x5jx2" Jul 28 17:11:37.670546 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.670532 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/359826bf-3f3a-4acc-8061-1ad4ad5dd832-must-gather-output\") pod \"must-gather-x5jx2\" (UID: \"359826bf-3f3a-4acc-8061-1ad4ad5dd832\") " pod="openshift-must-gather-rw7zh/must-gather-x5jx2" Jul 28 17:11:37.680656 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.680620 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgqcv\" (UniqueName: \"kubernetes.io/projected/359826bf-3f3a-4acc-8061-1ad4ad5dd832-kube-api-access-wgqcv\") pod \"must-gather-x5jx2\" (UID: \"359826bf-3f3a-4acc-8061-1ad4ad5dd832\") " pod="openshift-must-gather-rw7zh/must-gather-x5jx2" Jul 28 17:11:37.828200 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.828108 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rw7zh/must-gather-x5jx2" Jul 28 17:11:37.967113 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.967087 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rw7zh/must-gather-x5jx2"] Jul 28 17:11:37.973079 ip-10-0-140-22 kubenswrapper[2566]: W0728 17:11:37.973039 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod359826bf_3f3a_4acc_8061_1ad4ad5dd832.slice/crio-9f2496e8f7b0dd13eff9522a23fae547def485ae8c83c55f9660779bdff5c5b2 WatchSource:0}: Error finding container 9f2496e8f7b0dd13eff9522a23fae547def485ae8c83c55f9660779bdff5c5b2: Status 404 returned error can't find the container with id 9f2496e8f7b0dd13eff9522a23fae547def485ae8c83c55f9660779bdff5c5b2 Jul 28 17:11:37.974449 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:37.974434 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:11:38.828090 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:38.828053 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rw7zh/must-gather-x5jx2" event={"ID":"359826bf-3f3a-4acc-8061-1ad4ad5dd832","Type":"ContainerStarted","Data":"9f2496e8f7b0dd13eff9522a23fae547def485ae8c83c55f9660779bdff5c5b2"} Jul 28 17:11:44.859510 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:44.859425 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rw7zh/must-gather-x5jx2" event={"ID":"359826bf-3f3a-4acc-8061-1ad4ad5dd832","Type":"ContainerStarted","Data":"78b117530856484bd5a032effb4bda63ddd2d15ddbd2a0f4ff7b2b4b04a13822"} Jul 28 17:11:44.859510 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:44.859474 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rw7zh/must-gather-x5jx2" event={"ID":"359826bf-3f3a-4acc-8061-1ad4ad5dd832","Type":"ContainerStarted","Data":"bba7f4edc6d8890fb5e34bd9e67d79678536d88859cedcdaecc68cab0fca685c"} Jul 28 17:11:44.877107 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:11:44.877053 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rw7zh/must-gather-x5jx2" podStartSLOduration=1.503379357 podStartE2EDuration="7.877030826s" totalImagesPullingTime="6.373651469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:44.875069432 +0000 UTC m=+3046.257259091" watchObservedRunningTime="2026-07-28 17:11:44.877030826 +0000 UTC m=+3046.259220443" Jul 28 17:12:06.965963 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:06.965928 2566 generic.go:356] "Generic (PLEG): container finished" podID="359826bf-3f3a-4acc-8061-1ad4ad5dd832" containerID="bba7f4edc6d8890fb5e34bd9e67d79678536d88859cedcdaecc68cab0fca685c" exitCode=0 Jul 28 17:12:06.966402 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:06.966013 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rw7zh/must-gather-x5jx2" event={"ID":"359826bf-3f3a-4acc-8061-1ad4ad5dd832","Type":"ContainerDied","Data":"bba7f4edc6d8890fb5e34bd9e67d79678536d88859cedcdaecc68cab0fca685c"} Jul 28 17:12:06.966402 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:06.966352 2566 scope.go:117] "RemoveContainer" containerID="bba7f4edc6d8890fb5e34bd9e67d79678536d88859cedcdaecc68cab0fca685c" Jul 28 17:12:07.038488 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:07.038455 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-rw7zh_must-gather-x5jx2_359826bf-3f3a-4acc-8061-1ad4ad5dd832/gather/0.log" Jul 28 17:12:10.337309 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:10.337273 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nzcls_02ce611d-aef7-4996-bb0f-80279da441b5/global-pull-secret-syncer/0.log" Jul 28 17:12:10.469425 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:10.469395 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-h54gg_f6fa7f51-199b-4804-a481-e8641bad9f5b/konnectivity-agent/0.log" Jul 28 17:12:10.545335 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:10.545311 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-22.ec2.internal_c8a16f2abd34aee43b0a0ab3f7f0846e/haproxy/0.log" Jul 28 17:12:12.459543 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:12.459500 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-rw7zh/must-gather-x5jx2"] Jul 28 17:12:12.460074 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:12.459697 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-rw7zh/must-gather-x5jx2" podUID="359826bf-3f3a-4acc-8061-1ad4ad5dd832" containerName="copy" containerID="cri-o://78b117530856484bd5a032effb4bda63ddd2d15ddbd2a0f4ff7b2b4b04a13822" gracePeriod=2 Jul 28 17:12:12.462610 ip-10-0-140-22 kubenswrapper[2566]: E0728 17:12:12.462568 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-x5jx2\" is forbidden: User \"system:node:ip-10-0-140-22.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-rw7zh\": no relationship found between node 'ip-10-0-140-22.ec2.internal' and this object" podUID="359826bf-3f3a-4acc-8061-1ad4ad5dd832" pod="openshift-must-gather-rw7zh/must-gather-x5jx2" Jul 28 17:12:12.464469 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:12.463855 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-rw7zh/must-gather-x5jx2"] Jul 28 17:12:12.695935 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:12.695910 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-rw7zh_must-gather-x5jx2_359826bf-3f3a-4acc-8061-1ad4ad5dd832/copy/0.log" Jul 28 17:12:12.698550 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:12.698530 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rw7zh/must-gather-x5jx2" Jul 28 17:12:12.700928 ip-10-0-140-22 kubenswrapper[2566]: E0728 17:12:12.700905 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-x5jx2\" is forbidden: User \"system:node:ip-10-0-140-22.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-rw7zh\": no relationship found between node 'ip-10-0-140-22.ec2.internal' and this object" podUID="359826bf-3f3a-4acc-8061-1ad4ad5dd832" pod="openshift-must-gather-rw7zh/must-gather-x5jx2" Jul 28 17:12:12.804483 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:12.804452 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgqcv\" (UniqueName: \"kubernetes.io/projected/359826bf-3f3a-4acc-8061-1ad4ad5dd832-kube-api-access-wgqcv\") pod \"359826bf-3f3a-4acc-8061-1ad4ad5dd832\" (UID: \"359826bf-3f3a-4acc-8061-1ad4ad5dd832\") " Jul 28 17:12:12.804661 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:12.804506 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/359826bf-3f3a-4acc-8061-1ad4ad5dd832-must-gather-output\") pod \"359826bf-3f3a-4acc-8061-1ad4ad5dd832\" (UID: \"359826bf-3f3a-4acc-8061-1ad4ad5dd832\") " Jul 28 17:12:12.806032 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:12.806005 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/359826bf-3f3a-4acc-8061-1ad4ad5dd832-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "359826bf-3f3a-4acc-8061-1ad4ad5dd832" (UID: "359826bf-3f3a-4acc-8061-1ad4ad5dd832"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:12:12.806902 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:12.806879 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/359826bf-3f3a-4acc-8061-1ad4ad5dd832-kube-api-access-wgqcv" (OuterVolumeSpecName: "kube-api-access-wgqcv") pod "359826bf-3f3a-4acc-8061-1ad4ad5dd832" (UID: "359826bf-3f3a-4acc-8061-1ad4ad5dd832"). InnerVolumeSpecName "kube-api-access-wgqcv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:12:12.905631 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:12.905594 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgqcv\" (UniqueName: \"kubernetes.io/projected/359826bf-3f3a-4acc-8061-1ad4ad5dd832-kube-api-access-wgqcv\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 17:12:12.905631 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:12.905625 2566 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/359826bf-3f3a-4acc-8061-1ad4ad5dd832-must-gather-output\") on node \"ip-10-0-140-22.ec2.internal\" DevicePath \"\"" Jul 28 17:12:12.995037 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:12.995004 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-rw7zh_must-gather-x5jx2_359826bf-3f3a-4acc-8061-1ad4ad5dd832/copy/0.log" Jul 28 17:12:12.995300 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:12.995278 2566 generic.go:356] "Generic (PLEG): container finished" podID="359826bf-3f3a-4acc-8061-1ad4ad5dd832" containerID="78b117530856484bd5a032effb4bda63ddd2d15ddbd2a0f4ff7b2b4b04a13822" exitCode=143 Jul 28 17:12:12.995376 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:12.995323 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rw7zh/must-gather-x5jx2" Jul 28 17:12:12.995419 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:12.995392 2566 scope.go:117] "RemoveContainer" containerID="78b117530856484bd5a032effb4bda63ddd2d15ddbd2a0f4ff7b2b4b04a13822" Jul 28 17:12:12.997724 ip-10-0-140-22 kubenswrapper[2566]: E0728 17:12:12.997695 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-x5jx2\" is forbidden: User \"system:node:ip-10-0-140-22.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-rw7zh\": no relationship found between node 'ip-10-0-140-22.ec2.internal' and this object" podUID="359826bf-3f3a-4acc-8061-1ad4ad5dd832" pod="openshift-must-gather-rw7zh/must-gather-x5jx2" Jul 28 17:12:13.003945 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:13.003926 2566 scope.go:117] "RemoveContainer" containerID="bba7f4edc6d8890fb5e34bd9e67d79678536d88859cedcdaecc68cab0fca685c" Jul 28 17:12:13.005439 ip-10-0-140-22 kubenswrapper[2566]: E0728 17:12:13.005410 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-x5jx2\" is forbidden: User \"system:node:ip-10-0-140-22.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-rw7zh\": no relationship found between node 'ip-10-0-140-22.ec2.internal' and this object" podUID="359826bf-3f3a-4acc-8061-1ad4ad5dd832" pod="openshift-must-gather-rw7zh/must-gather-x5jx2" Jul 28 17:12:13.015822 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:13.015800 2566 scope.go:117] "RemoveContainer" containerID="78b117530856484bd5a032effb4bda63ddd2d15ddbd2a0f4ff7b2b4b04a13822" Jul 28 17:12:13.016209 ip-10-0-140-22 kubenswrapper[2566]: E0728 17:12:13.016177 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78b117530856484bd5a032effb4bda63ddd2d15ddbd2a0f4ff7b2b4b04a13822\": container with ID starting with 78b117530856484bd5a032effb4bda63ddd2d15ddbd2a0f4ff7b2b4b04a13822 not found: ID does not exist" containerID="78b117530856484bd5a032effb4bda63ddd2d15ddbd2a0f4ff7b2b4b04a13822" Jul 28 17:12:13.016317 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:13.016223 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78b117530856484bd5a032effb4bda63ddd2d15ddbd2a0f4ff7b2b4b04a13822"} err="failed to get container status \"78b117530856484bd5a032effb4bda63ddd2d15ddbd2a0f4ff7b2b4b04a13822\": rpc error: code = NotFound desc = could not find container \"78b117530856484bd5a032effb4bda63ddd2d15ddbd2a0f4ff7b2b4b04a13822\": container with ID starting with 78b117530856484bd5a032effb4bda63ddd2d15ddbd2a0f4ff7b2b4b04a13822 not found: ID does not exist" Jul 28 17:12:13.016317 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:13.016246 2566 scope.go:117] "RemoveContainer" containerID="bba7f4edc6d8890fb5e34bd9e67d79678536d88859cedcdaecc68cab0fca685c" Jul 28 17:12:13.016517 ip-10-0-140-22 kubenswrapper[2566]: E0728 17:12:13.016500 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bba7f4edc6d8890fb5e34bd9e67d79678536d88859cedcdaecc68cab0fca685c\": container with ID starting with bba7f4edc6d8890fb5e34bd9e67d79678536d88859cedcdaecc68cab0fca685c not found: ID does not exist" containerID="bba7f4edc6d8890fb5e34bd9e67d79678536d88859cedcdaecc68cab0fca685c" Jul 28 17:12:13.016568 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:13.016522 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bba7f4edc6d8890fb5e34bd9e67d79678536d88859cedcdaecc68cab0fca685c"} err="failed to get container status \"bba7f4edc6d8890fb5e34bd9e67d79678536d88859cedcdaecc68cab0fca685c\": rpc error: code = NotFound desc = could not find container \"bba7f4edc6d8890fb5e34bd9e67d79678536d88859cedcdaecc68cab0fca685c\": container with ID starting with bba7f4edc6d8890fb5e34bd9e67d79678536d88859cedcdaecc68cab0fca685c not found: ID does not exist" Jul 28 17:12:13.107098 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:13.107012 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="359826bf-3f3a-4acc-8061-1ad4ad5dd832" path="/var/lib/kubelet/pods/359826bf-3f3a-4acc-8061-1ad4ad5dd832/volumes" Jul 28 17:12:14.183470 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:14.183438 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f3f67baf-e5be-4222-b890-cc2fc9b3f5f5/alertmanager/0.log" Jul 28 17:12:14.228988 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:14.228942 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f3f67baf-e5be-4222-b890-cc2fc9b3f5f5/config-reloader/0.log" Jul 28 17:12:14.265240 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:14.265216 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f3f67baf-e5be-4222-b890-cc2fc9b3f5f5/kube-rbac-proxy-web/0.log" Jul 28 17:12:14.301253 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:14.301223 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f3f67baf-e5be-4222-b890-cc2fc9b3f5f5/kube-rbac-proxy/0.log" Jul 28 17:12:14.338185 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:14.338161 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f3f67baf-e5be-4222-b890-cc2fc9b3f5f5/kube-rbac-proxy-metric/0.log" Jul 28 17:12:14.370163 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:14.370137 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f3f67baf-e5be-4222-b890-cc2fc9b3f5f5/prom-label-proxy/0.log" Jul 28 17:12:14.410779 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:14.410734 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f3f67baf-e5be-4222-b890-cc2fc9b3f5f5/init-config-reloader/0.log" Jul 28 17:12:14.456709 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:14.456611 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-fz4bt_54e904f0-e7ab-4882-9b0d-a91248f578a4/cluster-monitoring-operator/0.log" Jul 28 17:12:14.585067 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:14.585036 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7b9878b988-mgx2k_0020da7d-430b-4e21-9acb-737c6e75b06e/metrics-server/0.log" Jul 28 17:12:14.618486 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:14.618447 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-j5jn5_179cd673-52a8-4c1d-97e4-88ee48ac0483/monitoring-plugin/0.log" Jul 28 17:12:14.816597 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:14.816571 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ts6kf_19611934-daaf-47b3-b63f-2159cb2d7f69/node-exporter/0.log" Jul 28 17:12:14.845102 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:14.845077 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ts6kf_19611934-daaf-47b3-b63f-2159cb2d7f69/kube-rbac-proxy/0.log" Jul 28 17:12:14.874326 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:14.874301 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ts6kf_19611934-daaf-47b3-b63f-2159cb2d7f69/init-textfile/0.log" Jul 28 17:12:15.020513 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:15.020486 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_252febaf-ea3e-4cc0-be7f-d287f1a2f9a9/prometheus/0.log" Jul 28 17:12:15.041372 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:15.041340 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_252febaf-ea3e-4cc0-be7f-d287f1a2f9a9/config-reloader/0.log" Jul 28 17:12:15.070380 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:15.070305 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_252febaf-ea3e-4cc0-be7f-d287f1a2f9a9/thanos-sidecar/0.log" Jul 28 17:12:15.098956 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:15.098930 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_252febaf-ea3e-4cc0-be7f-d287f1a2f9a9/kube-rbac-proxy-web/0.log" Jul 28 17:12:15.122918 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:15.122889 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_252febaf-ea3e-4cc0-be7f-d287f1a2f9a9/kube-rbac-proxy/0.log" Jul 28 17:12:15.149922 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:15.149892 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_252febaf-ea3e-4cc0-be7f-d287f1a2f9a9/kube-rbac-proxy-thanos/0.log" Jul 28 17:12:15.175342 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:15.175306 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_252febaf-ea3e-4cc0-be7f-d287f1a2f9a9/init-config-reloader/0.log" Jul 28 17:12:15.209492 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:15.209463 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-cqgvf_29fdb1e7-dba8-4116-bd0b-a28d038bdf39/prometheus-operator/0.log" Jul 28 17:12:15.231315 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:15.231283 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-cqgvf_29fdb1e7-dba8-4116-bd0b-a28d038bdf39/kube-rbac-proxy/0.log" Jul 28 17:12:15.256108 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:15.256075 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-kds9d_53c435c8-0bb5-464f-8ef0-39483cd69209/prometheus-operator-admission-webhook/0.log" Jul 28 17:12:17.443890 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:17.443830 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/2.log" Jul 28 17:12:17.448479 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:17.448458 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-fcmgz_ed0c86cb-26a6-4633-9170-9a042a3a71b3/console-operator/3.log" Jul 28 17:12:18.410382 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.410351 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64"] Jul 28 17:12:18.410666 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.410654 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="359826bf-3f3a-4acc-8061-1ad4ad5dd832" containerName="gather" Jul 28 17:12:18.410715 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.410668 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="359826bf-3f3a-4acc-8061-1ad4ad5dd832" containerName="gather" Jul 28 17:12:18.410715 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.410677 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="359826bf-3f3a-4acc-8061-1ad4ad5dd832" containerName="copy" Jul 28 17:12:18.410715 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.410683 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="359826bf-3f3a-4acc-8061-1ad4ad5dd832" containerName="copy" Jul 28 17:12:18.410810 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.410748 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="359826bf-3f3a-4acc-8061-1ad4ad5dd832" containerName="copy" Jul 28 17:12:18.410810 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.410756 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="359826bf-3f3a-4acc-8061-1ad4ad5dd832" containerName="gather" Jul 28 17:12:18.415751 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.415725 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.419147 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.419120 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-464v8\"/\"kube-root-ca.crt\"" Jul 28 17:12:18.419990 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.419958 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-464v8\"/\"openshift-service-ca.crt\"" Jul 28 17:12:18.420100 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.420059 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-464v8\"/\"default-dockercfg-mdmkx\"" Jul 28 17:12:18.424507 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.424480 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64"] Jul 28 17:12:18.425721 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.425689 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-2m7fh_ce70be22-819d-4675-9c98-380ffd733f4c/volume-data-source-validator/0.log" Jul 28 17:12:18.454176 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.454142 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8-lib-modules\") pod \"perf-node-gather-daemonset-vbl64\" (UID: \"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.454176 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.454179 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8-podres\") pod \"perf-node-gather-daemonset-vbl64\" (UID: \"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.454671 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.454211 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnwk4\" (UniqueName: \"kubernetes.io/projected/4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8-kube-api-access-hnwk4\") pod \"perf-node-gather-daemonset-vbl64\" (UID: \"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.454671 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.454335 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8-proc\") pod \"perf-node-gather-daemonset-vbl64\" (UID: \"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.454671 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.454405 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8-sys\") pod \"perf-node-gather-daemonset-vbl64\" (UID: \"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.555587 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.555544 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8-proc\") pod \"perf-node-gather-daemonset-vbl64\" (UID: \"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.555804 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.555601 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8-sys\") pod \"perf-node-gather-daemonset-vbl64\" (UID: \"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.555804 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.555637 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8-lib-modules\") pod \"perf-node-gather-daemonset-vbl64\" (UID: \"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.555804 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.555655 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8-podres\") pod \"perf-node-gather-daemonset-vbl64\" (UID: \"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.555804 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.555675 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnwk4\" (UniqueName: \"kubernetes.io/projected/4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8-kube-api-access-hnwk4\") pod \"perf-node-gather-daemonset-vbl64\" (UID: \"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.555804 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.555698 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8-proc\") pod \"perf-node-gather-daemonset-vbl64\" (UID: \"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.555804 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.555760 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8-sys\") pod \"perf-node-gather-daemonset-vbl64\" (UID: \"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.556158 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.555808 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8-lib-modules\") pod \"perf-node-gather-daemonset-vbl64\" (UID: \"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.556158 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.555832 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8-podres\") pod \"perf-node-gather-daemonset-vbl64\" (UID: \"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.564241 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.564217 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnwk4\" (UniqueName: \"kubernetes.io/projected/4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8-kube-api-access-hnwk4\") pod \"perf-node-gather-daemonset-vbl64\" (UID: \"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8\") " pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.727737 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.727633 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:18.862739 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:18.862714 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64"] Jul 28 17:12:18.865297 ip-10-0-140-22 kubenswrapper[2566]: W0728 17:12:18.865266 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4fbb4ea3_bb9d_4be7_a2d3_4f7bb419f7e8.slice/crio-97e440e30051f7fe6feb95ff61e0b24366d8093ee9777c0bba696b3143934075 WatchSource:0}: Error finding container 97e440e30051f7fe6feb95ff61e0b24366d8093ee9777c0bba696b3143934075: Status 404 returned error can't find the container with id 97e440e30051f7fe6feb95ff61e0b24366d8093ee9777c0bba696b3143934075 Jul 28 17:12:19.024868 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:19.024837 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" event={"ID":"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8","Type":"ContainerStarted","Data":"99da363d4a96ac75e282d68e65109dfe0f82e71d0f5b23b9e25069c8e9f47441"} Jul 28 17:12:19.025095 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:19.024886 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" event={"ID":"4fbb4ea3-bb9d-4be7-a2d3-4f7bb419f7e8","Type":"ContainerStarted","Data":"97e440e30051f7fe6feb95ff61e0b24366d8093ee9777c0bba696b3143934075"} Jul 28 17:12:19.025095 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:19.024992 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:19.043195 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:19.043135 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" podStartSLOduration=1.04311772 podStartE2EDuration="1.04311772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:12:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:12:19.041088438 +0000 UTC m=+3080.423278057" watchObservedRunningTime="2026-07-28 17:12:19.04311772 +0000 UTC m=+3080.425307337" Jul 28 17:12:19.394411 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:19.394315 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fdhkh_08c6af7a-3f2a-491b-aee3-61a1dafe2c0a/dns/0.log" Jul 28 17:12:19.425574 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:19.425545 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fdhkh_08c6af7a-3f2a-491b-aee3-61a1dafe2c0a/kube-rbac-proxy/0.log" Jul 28 17:12:19.593907 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:19.593880 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8fpkf_4d2caa30-ece8-42cf-9dc4-e45e56dae4a2/dns-node-resolver/0.log" Jul 28 17:12:20.135441 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:20.135409 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6jp82_18769dd9-4ec7-45b1-994f-6d0d3bd4aa8b/node-ca/0.log" Jul 28 17:12:20.923037 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:20.923006 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-855657b557-78fr5_cab67026-e646-4a71-9c1a-e21d1fda8536/router/0.log" Jul 28 17:12:21.305543 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:21.305516 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tftts_7a43fcfe-07e8-4f38-bfff-fb4cdedb6d3b/serve-healthcheck-canary/0.log" Jul 28 17:12:21.674429 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:21.674350 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-9hwqx_676a1e19-6f6c-42a4-881c-730a584c1c2b/insights-operator/0.log" Jul 28 17:12:21.674931 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:21.674916 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-9hwqx_676a1e19-6f6c-42a4-881c-730a584c1c2b/insights-operator/1.log" Jul 28 17:12:21.756907 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:21.756885 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-chq49_63243f80-5a2a-4991-91e4-3f217aa9c2cb/kube-rbac-proxy/0.log" Jul 28 17:12:21.778380 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:21.778354 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-chq49_63243f80-5a2a-4991-91e4-3f217aa9c2cb/exporter/0.log" Jul 28 17:12:21.800022 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:21.799990 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-chq49_63243f80-5a2a-4991-91e4-3f217aa9c2cb/extractor/0.log" Jul 28 17:12:24.248141 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:24.248109 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-clpcc_ffb26a63-5ce3-40e8-991b-b997e95c6e0f/s3-init/0.log" Jul 28 17:12:24.265170 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:24.265141 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-clpcc_ffb26a63-5ce3-40e8-991b-b997e95c6e0f/download-hf-model/0.log" Jul 28 17:12:25.040256 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:25.040223 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-464v8/perf-node-gather-daemonset-vbl64" Jul 28 17:12:28.764691 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:28.764657 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n_51fb28cb-f191-418e-ae54-43ca310712ec/kube-storage-version-migrator-operator/1.log" Jul 28 17:12:28.765571 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:28.765554 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-ddm8n_51fb28cb-f191-418e-ae54-43ca310712ec/kube-storage-version-migrator-operator/0.log" Jul 28 17:12:29.755365 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:29.755337 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-knhs8_faeb9e49-9c13-4f71-9e6c-b14e68624005/kube-multus-additional-cni-plugins/0.log" Jul 28 17:12:29.781906 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:29.781875 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-knhs8_faeb9e49-9c13-4f71-9e6c-b14e68624005/egress-router-binary-copy/0.log" Jul 28 17:12:29.803130 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:29.803097 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-knhs8_faeb9e49-9c13-4f71-9e6c-b14e68624005/cni-plugins/0.log" Jul 28 17:12:29.824044 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:29.824005 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-knhs8_faeb9e49-9c13-4f71-9e6c-b14e68624005/bond-cni-plugin/0.log" Jul 28 17:12:29.846584 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:29.846547 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-knhs8_faeb9e49-9c13-4f71-9e6c-b14e68624005/routeoverride-cni/0.log" Jul 28 17:12:29.874012 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:29.873961 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-knhs8_faeb9e49-9c13-4f71-9e6c-b14e68624005/whereabouts-cni-bincopy/0.log" Jul 28 17:12:29.898685 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:29.898655 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-knhs8_faeb9e49-9c13-4f71-9e6c-b14e68624005/whereabouts-cni/0.log" Jul 28 17:12:30.286759 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:30.286730 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lhg5z_312acc96-6bbf-4d5e-98fd-f5142efd2bdd/kube-multus/0.log" Jul 28 17:12:30.309915 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:30.309886 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gb4cw_4a8f2962-0386-4066-a462-445cbb6c3f09/network-metrics-daemon/0.log" Jul 28 17:12:30.328614 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:30.328585 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-gb4cw_4a8f2962-0386-4066-a462-445cbb6c3f09/kube-rbac-proxy/0.log" Jul 28 17:12:31.542484 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:31.542455 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-controller/0.log" Jul 28 17:12:31.564070 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:31.564044 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/0.log" Jul 28 17:12:31.564226 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:31.564133 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovn-acl-logging/1.log" Jul 28 17:12:31.582783 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:31.582751 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/kube-rbac-proxy-node/0.log" Jul 28 17:12:31.604959 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:31.604928 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/kube-rbac-proxy-ovn-metrics/0.log" Jul 28 17:12:31.625275 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:31.625248 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/northd/0.log" Jul 28 17:12:31.649565 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:31.649534 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/nbdb/0.log" Jul 28 17:12:31.668767 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:31.668743 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/sbdb/0.log" Jul 28 17:12:31.708693 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:31.708658 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xx7xl_54b3868a-68e2-408f-90a8-808c53e0aefe/ovnkube-controller/0.log" Jul 28 17:12:32.666800 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:32.666774 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-gbdb4_6d6d1243-b8d6-4402-96f1-5350263231ab/network-check-target-container/0.log" Jul 28 17:12:33.590942 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:33.590914 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-rvhjb_c72d215a-f550-422a-98db-d21bedaca835/iptables-alerter/0.log" Jul 28 17:12:34.194213 ip-10-0-140-22 kubenswrapper[2566]: I0728 17:12:34.194180 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-82p66_4258901a-a0ea-48aa-921d-a3217ab8a119/tuned/0.log"