Jul 09 03:59:51.018120 ip-10-0-137-39 systemd[1]: Starting Kubernetes Kubelet... Jul 09 03:59:51.434879 ip-10-0-137-39 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 03:59:51.434879 ip-10-0-137-39 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 03:59:51.434879 ip-10-0-137-39 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 03:59:51.434879 ip-10-0-137-39 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 03:59:51.434879 ip-10-0-137-39 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 03:59:51.434879 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.355745 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 03:59:51.434879 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360118 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 03:59:51.434879 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360130 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 03:59:51.434879 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360134 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 03:59:51.434879 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360137 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 03:59:51.434879 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360141 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 03:59:51.434879 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360144 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 03:59:51.434879 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360148 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 03:59:51.434879 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360161 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360166 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360170 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360173 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360175 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360178 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360181 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360184 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360187 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360190 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360193 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360197 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360199 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360202 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360205 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360209 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360211 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360215 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360218 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360221 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 03:59:51.435792 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360224 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360226 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360229 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360232 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360235 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360238 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360240 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360243 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360246 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360249 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360251 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360254 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360257 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360260 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360263 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360266 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360268 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360271 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360273 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360276 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 03:59:51.436779 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360279 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360281 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360284 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360288 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360291 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360295 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360297 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360300 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360304 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360306 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360309 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360312 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360315 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360318 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360321 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360323 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360326 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360329 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360332 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360334 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 03:59:51.437405 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360337 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360340 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360343 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360347 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360352 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360355 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360359 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360362 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360365 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360367 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360370 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360373 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360377 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360381 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360383 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360387 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360390 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360393 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360395 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 03:59:51.438028 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360401 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360404 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360408 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360412 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360416 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360420 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360423 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360428 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360431 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360434 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360438 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360441 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.360443 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361214 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361220 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361223 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361226 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361229 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361233 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 03:59:51.438715 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361235 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361239 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361242 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361244 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361247 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361250 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361253 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361257 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361261 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361264 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361268 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361270 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361273 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361276 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361279 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361282 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361284 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361287 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361290 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 03:59:51.439324 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361292 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361295 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361298 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361301 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361303 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361306 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361309 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361312 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361315 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361317 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361320 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361323 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361325 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361329 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361331 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361335 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361338 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361340 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361343 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361346 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 03:59:51.440215 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361349 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361351 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361354 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361357 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361360 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361362 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361365 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361368 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361371 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361374 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361376 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361379 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361382 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361384 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361387 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361390 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361392 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361395 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361398 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361402 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 03:59:51.440842 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361405 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361407 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361410 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361413 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361415 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361418 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361421 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361423 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361427 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361429 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361432 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361435 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361438 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361441 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361443 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361446 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361449 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361452 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361455 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 03:59:51.608634 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361458 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 03:59:51.454042 ip-10-0-137-39 systemd[1]: Started Kubernetes Kubelet. Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361463 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361467 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361470 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361473 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361477 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361480 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361483 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361486 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361489 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361492 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361500 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361504 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361507 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361510 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361902 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361907 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361910 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361935 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 03:59:51.636313 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361938 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361942 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361946 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361949 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361953 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361955 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361958 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361962 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361964 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361967 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361970 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361973 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361976 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361979 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361981 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361984 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361987 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361989 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361992 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 03:59:51.811426 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361995 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.361998 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362002 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362004 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362007 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362013 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362016 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362018 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362021 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362024 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362027 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362029 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362032 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362035 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362038 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362040 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362043 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362046 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362049 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 03:59:51.812880 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362052 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362055 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362058 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362061 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362064 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362066 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362069 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362072 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362075 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362077 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362080 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362083 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362086 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362088 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362091 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362094 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362097 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362101 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362107 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362110 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 03:59:51.813855 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362117 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362121 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362124 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362127 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362129 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362132 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362135 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362138 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362140 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362144 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362147 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362150 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362153 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362157 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362160 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362163 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362166 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362169 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362172 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362175 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 03:59:51.815471 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362177 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362180 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362183 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362186 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362188 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362191 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362194 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362197 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362200 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362203 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362209 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362211 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362216 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362218 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362222 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362224 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.362227 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362587 2576 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362659 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362670 2576 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362678 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 03:59:51.816435 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362686 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362692 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362701 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362716 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362721 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362727 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362733 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362739 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362744 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362750 2576 flags.go:64] FLAG: --cgroup-root="" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362755 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362761 2576 flags.go:64] FLAG: --client-ca-file="" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362770 2576 flags.go:64] FLAG: --cloud-provider="external" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362776 2576 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362784 2576 flags.go:64] FLAG: --cluster-domain="" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362789 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362794 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362800 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362806 2576 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362813 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362823 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362832 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362838 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362845 2576 flags.go:64] FLAG: --contention-profiling="false" Jul 09 03:59:51.817834 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362850 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362856 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362861 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362866 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362874 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362902 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362908 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362915 2576 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362920 2576 flags.go:64] FLAG: --enable-server="true" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362926 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362934 2576 flags.go:64] FLAG: --event-burst="100" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362940 2576 flags.go:64] FLAG: --event-qps="50" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362946 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362956 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362962 2576 flags.go:64] FLAG: --eviction-hard="" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362968 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362975 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362983 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362991 2576 flags.go:64] FLAG: --eviction-soft="" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.362999 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363004 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363012 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363026 2576 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363034 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363041 2576 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 03:59:51.819039 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363049 2576 flags.go:64] FLAG: --feature-gates="" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363058 2576 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363066 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363073 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363084 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363100 2576 flags.go:64] FLAG: --healthz-port="10248" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363109 2576 flags.go:64] FLAG: --help="false" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363117 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-137-39.ec2.internal" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363124 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363132 2576 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363139 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363148 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363157 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363163 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363173 2576 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363178 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363184 2576 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363190 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363196 2576 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363202 2576 flags.go:64] FLAG: --kube-reserved="" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363208 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363214 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363225 2576 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363230 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 03:59:51.820678 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363236 2576 flags.go:64] FLAG: --lock-file="" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363241 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363247 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363253 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363263 2576 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363268 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363273 2576 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363283 2576 flags.go:64] FLAG: --logging-format="text" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363289 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363295 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363300 2576 flags.go:64] FLAG: --manifest-url="" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363305 2576 flags.go:64] FLAG: --manifest-url-header="" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363315 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363320 2576 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363330 2576 flags.go:64] FLAG: --max-pods="110" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363341 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363346 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363352 2576 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363357 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363362 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363367 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363372 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363392 2576 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363397 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363402 2576 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 03:59:51.822762 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363408 2576 flags.go:64] FLAG: --pod-cidr="" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363413 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363423 2576 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363429 2576 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363435 2576 flags.go:64] FLAG: --pods-per-core="0" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363439 2576 flags.go:64] FLAG: --port="10250" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363449 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363455 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-001916450b9801038" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363460 2576 flags.go:64] FLAG: --qos-reserved="" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363466 2576 flags.go:64] FLAG: --read-only-port="10255" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363471 2576 flags.go:64] FLAG: --register-node="true" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363477 2576 flags.go:64] FLAG: --register-with-taints="" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363483 2576 flags.go:64] FLAG: --registry-burst="10" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363488 2576 flags.go:64] FLAG: --registry-qps="5" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363499 2576 flags.go:64] FLAG: --reserved-cpus="" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363503 2576 flags.go:64] FLAG: --reserved-memory="" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363510 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363516 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363521 2576 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363528 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363533 2576 flags.go:64] FLAG: --runonce="false" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363538 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363550 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363556 2576 flags.go:64] FLAG: --seccomp-default="false" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363561 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 03:59:51.823938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363566 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363571 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363577 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363582 2576 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363588 2576 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363593 2576 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363603 2576 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363608 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363614 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363619 2576 flags.go:64] FLAG: --system-cgroups="" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363624 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363636 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363641 2576 flags.go:64] FLAG: --tls-cert-file="" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363652 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363661 2576 flags.go:64] FLAG: --tls-min-version="" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363673 2576 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363678 2576 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363682 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363695 2576 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363701 2576 flags.go:64] FLAG: --v="2" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363709 2576 flags.go:64] FLAG: --version="false" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363716 2576 flags.go:64] FLAG: --vmodule="" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363722 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.363733 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364250 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 03:59:51.825040 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364266 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364270 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364274 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364284 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364287 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364291 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364294 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364297 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364317 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364320 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364469 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364472 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364475 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364478 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364481 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364485 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364488 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364491 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364493 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364496 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 03:59:51.826188 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364499 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364503 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364506 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364508 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364512 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364514 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364518 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364521 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364524 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364527 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364530 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364533 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364537 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364540 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364542 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364545 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364548 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364551 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364554 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364556 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 03:59:51.827086 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364559 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364566 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364571 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364574 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364578 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364580 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364583 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364586 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364589 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364592 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364595 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364598 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364601 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364604 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364607 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364610 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364613 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364616 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364621 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 03:59:51.827888 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364624 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364630 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364635 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364638 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364643 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364646 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364648 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364651 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364655 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364659 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364662 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364665 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364668 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364672 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364676 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364680 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364683 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364686 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364689 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 03:59:51.828675 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364692 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364694 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364697 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364699 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364702 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364706 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364708 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364711 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364714 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364717 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364719 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364722 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364725 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364728 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364732 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364735 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364737 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364740 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364743 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 03:59:51.829491 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.364746 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 03:59:51.830270 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.365384 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 03:59:51.830270 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.372719 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 09 03:59:51.830270 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.372739 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 03:59:51.830270 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.372763 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 03:59:51.830270 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.372775 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 03:59:51.830270 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372840 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 03:59:51.830270 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372848 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 03:59:51.830270 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372851 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 03:59:51.830270 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372855 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 03:59:51.830270 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372859 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 03:59:51.830270 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372862 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 03:59:51.830270 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372866 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 03:59:51.830270 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372870 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 03:59:51.830270 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372874 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372879 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372882 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372887 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372891 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372895 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372900 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372903 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372906 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372931 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372936 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372940 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372943 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372947 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372951 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372955 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372958 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372963 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372965 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372968 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 03:59:51.830844 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372972 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372977 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372980 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372983 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372987 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372991 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.372994 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373003 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373007 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373010 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373013 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373016 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373020 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373023 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373026 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373030 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373033 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373036 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373040 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373044 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 03:59:51.831700 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373047 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373050 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373054 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373057 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373060 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373063 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373067 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373070 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373075 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373079 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373082 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373085 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373089 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373094 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373097 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373100 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373104 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373109 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373113 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 03:59:51.832527 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373117 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373120 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373123 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373127 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373132 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373135 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373138 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373142 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373145 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373148 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373151 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373154 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373157 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373160 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373165 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373169 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373172 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373175 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373179 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 03:59:51.833312 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373183 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 03:59:51.834078 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373186 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 03:59:51.834078 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373188 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 03:59:51.834078 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373192 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 03:59:51.834078 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373197 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 03:59:51.834078 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373200 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 03:59:51.834078 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373204 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 03:59:51.834078 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373209 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 03:59:51.834078 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373212 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 03:59:51.834078 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373215 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 03:59:51.834078 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373219 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 03:59:51.834078 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373221 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 03:59:51.834078 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373224 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 03:59:51.834078 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.373231 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 03:59:51.834078 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373363 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 03:59:51.834078 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373369 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373373 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373376 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373381 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373384 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373386 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373390 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373394 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373397 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373400 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373403 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373407 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373412 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373416 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373420 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373425 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373431 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373435 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373438 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 03:59:51.834493 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373442 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373446 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373451 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373456 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373459 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373464 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373467 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373471 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373474 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373477 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373481 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373484 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373486 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373489 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373492 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373495 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373497 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373501 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373504 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373507 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 03:59:51.834997 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373510 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373513 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373517 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373520 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373523 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373527 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373529 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373533 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373537 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373541 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373545 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373547 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373551 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373555 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373558 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373563 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373567 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373571 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373574 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 03:59:51.835526 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373578 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373581 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373584 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373587 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373592 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373595 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373598 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373601 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373605 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373608 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373610 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373614 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373618 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373621 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373623 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373627 2576 feature_gate.go:328] unrecognized feature gate: Example Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373630 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373633 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373636 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373639 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 03:59:51.836036 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373643 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373646 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373649 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373652 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373656 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373659 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373662 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373666 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373670 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373675 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373681 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373684 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373688 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373692 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373695 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373699 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373703 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373707 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373709 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 03:59:51.836544 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:51.373712 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 03:59:51.837127 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.373718 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 03:59:51.837127 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.373848 2576 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 03:59:51.837127 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.377223 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 03:59:51.837127 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.378512 2576 server.go:1019] "Starting client certificate rotation" Jul 09 03:59:51.837127 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.380758 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 03:59:51.837127 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.383647 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 03:59:51.837127 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.383675 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 03:59:51.837127 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.383670 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 03:59:51.837127 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.395612 2576 log.go:25] "Validated CRI v1 runtime API" Jul 09 03:59:51.837127 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.407527 2576 log.go:25] "Validated CRI v1 image API" Jul 09 03:59:51.837127 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.408548 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 03:59:51.837127 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.411483 2576 fs.go:135] Filesystem UUIDs: map[1c83a69a-194b-42b5-ab58-54349abab713:/dev/nvme0n1p3 6e8ff756-5597-4060-9d77-3c2974304c36:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 09 03:59:51.837458 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.411504 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 03:59:51.837493 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.425339 2576 manager.go:216] Machine: {Timestamp:2026-07-09 03:59:51.424131728 +0000 UTC m=+0.300049311 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3166538 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23e4bfc1c2dca29bd1a0aea7b95656 SystemUUID:ec23e4bf-c1c2-dca2-9bd1-a0aea7b95656 BootID:a771531f-0b69-491a-99b0-aaf5348ce73b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c2:74:bb:45:7b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c2:74:bb:45:7b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:a2:92:68:64:c1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 03:59:51.837493 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.425477 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 03:59:51.837493 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.425486 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 03:59:51.837493 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.425596 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 03:59:51.837493 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.428313 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 03:59:51.837493 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.428340 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-39.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 03:59:51.837493 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.429017 2576 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.429027 2576 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.429036 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.434552 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.435816 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.440059 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.440188 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.440218 2576 kubelet.go:486] "Attempting to sync node with API server" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.440227 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.440244 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.440254 2576 kubelet.go:398] "Adding apiserver pod source" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.440263 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.440645 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.440668 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.443747 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.445629 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.445666 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.446855 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.446874 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.446883 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.446891 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.446902 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.446910 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.446937 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.446946 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.446955 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.446962 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.446980 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.446986 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.448124 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.448134 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.450878 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-39.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.452847 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.452972 2576 server.go:1268] "Started kubelet" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.453073 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.453115 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-39.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.453220 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.453266 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.454980 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.456561 2576 server.go:310] "Adding debug handlers to kubelet server" Jul 09 03:59:51.837726 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.458786 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.464683 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-39.ec2.internal.18c08317fabdcb08 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-39.ec2.internal,UID:ip-10-0-137-39.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-39.ec2.internal,},FirstTimestamp:2026-07-09 03:59:51.45294516 +0000 UTC m=+0.328862741,LastTimestamp:2026-07-09 03:59:51.45294516 +0000 UTC m=+0.328862741,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-39.ec2.internal,}" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.466007 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.466021 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.466225 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-39.ec2.internal\" not found" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.466636 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.466648 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.466668 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.466791 2576 reconstruct.go:97] "Volume reconstruction finished" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.466801 2576 reconciler.go:29] "Reconciler: start to sync state" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.467332 2576 factory.go:55] Registering systemd factory Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.467353 2576 factory.go:223] Registration of the systemd container factory successfully Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.467697 2576 factory.go:153] Registering CRI-O factory Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.467715 2576 factory.go:223] Registration of the crio container factory successfully Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.467770 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.467806 2576 factory.go:103] Registering Raw factory Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.467827 2576 manager.go:1196] Started watching for new ooms in manager Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.468552 2576 manager.go:349] Starting recovery of all containers Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.472026 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-39.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.472219 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.477699 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cxxww" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.484866 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cxxww" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.488575 2576 manager.go:354] Recovery completed Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.493636 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.499497 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasSufficientMemory" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.499538 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.499549 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasSufficientPID" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.500253 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.500262 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.500273 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.500278 2576 state_mem.go:36] "Initialized new in-memory state store" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.501042 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasSufficientMemory" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.501070 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 03:59:51.838836 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.501088 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasSufficientPID" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.501227 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-39.ec2.internal.18c08317fd8480ad default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-39.ec2.internal,UID:ip-10-0-137-39.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-39.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-39.ec2.internal,},FirstTimestamp:2026-07-09 03:59:51.499522221 +0000 UTC m=+0.375439802,LastTimestamp:2026-07-09 03:59:51.499522221 +0000 UTC m=+0.375439802,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-39.ec2.internal,}" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.502494 2576 policy_none.go:49] "None policy: Start" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.502508 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.502519 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.509711 2576 policy_none.go:47] "Start" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.548561 2576 manager.go:337] "Starting Device Plugin manager" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.548775 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.548787 2576 server.go:87] "Starting device plugin registration server" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.549029 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.549042 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.549165 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.549291 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.549300 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.549865 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.549916 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-39.ec2.internal\" not found" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.616113 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.617437 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.617477 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.617501 2576 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.617539 2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.620259 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.649456 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.650473 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasSufficientMemory" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.650498 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.650509 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasSufficientPID" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.650534 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-39.ec2.internal" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.660364 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-39.ec2.internal" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.660386 2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-39.ec2.internal\": node \"ip-10-0-137-39.ec2.internal\" not found" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.688503 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-39.ec2.internal\" not found" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.718678 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-39.ec2.internal"] Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.718763 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.720416 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasSufficientMemory" Jul 09 03:59:51.839761 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.720447 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.720457 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasSufficientPID" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.722995 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.723182 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.723216 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.723759 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasSufficientMemory" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.723792 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.723802 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasSufficientPID" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.723761 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasSufficientMemory" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.723880 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.723895 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasSufficientPID" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.726279 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-39.ec2.internal" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.726303 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.727012 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasSufficientMemory" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.727030 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.727047 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeHasSufficientPID" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.763749 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-39.ec2.internal\" not found" node="ip-10-0-137-39.ec2.internal" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.768623 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/48c34b9b6bc4e55cf67844482ac41f2e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal\" (UID: \"48c34b9b6bc4e55cf67844482ac41f2e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.768649 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48c34b9b6bc4e55cf67844482ac41f2e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal\" (UID: \"48c34b9b6bc4e55cf67844482ac41f2e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.768685 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-39.ec2.internal\" not found" node="ip-10-0-137-39.ec2.internal" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.768685 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cabda21fc7b0aa2caee43190e64a6bed-config\") pod \"kube-apiserver-proxy-ip-10-0-137-39.ec2.internal\" (UID: \"cabda21fc7b0aa2caee43190e64a6bed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-39.ec2.internal" Jul 09 03:59:51.840692 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.788790 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-39.ec2.internal\" not found" Jul 09 03:59:51.869526 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.869455 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48c34b9b6bc4e55cf67844482ac41f2e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal\" (UID: \"48c34b9b6bc4e55cf67844482ac41f2e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal" Jul 09 03:59:51.869526 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.869492 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cabda21fc7b0aa2caee43190e64a6bed-config\") pod \"kube-apiserver-proxy-ip-10-0-137-39.ec2.internal\" (UID: \"cabda21fc7b0aa2caee43190e64a6bed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-39.ec2.internal" Jul 09 03:59:51.869526 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.869513 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/48c34b9b6bc4e55cf67844482ac41f2e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal\" (UID: \"48c34b9b6bc4e55cf67844482ac41f2e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal" Jul 09 03:59:51.869688 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.869542 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/48c34b9b6bc4e55cf67844482ac41f2e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal\" (UID: \"48c34b9b6bc4e55cf67844482ac41f2e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal" Jul 09 03:59:51.869688 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.869559 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48c34b9b6bc4e55cf67844482ac41f2e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal\" (UID: \"48c34b9b6bc4e55cf67844482ac41f2e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal" Jul 09 03:59:51.869688 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:51.869572 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/cabda21fc7b0aa2caee43190e64a6bed-config\") pod \"kube-apiserver-proxy-ip-10-0-137-39.ec2.internal\" (UID: \"cabda21fc7b0aa2caee43190e64a6bed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-39.ec2.internal" Jul 09 03:59:51.889100 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.889068 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-39.ec2.internal\" not found" Jul 09 03:59:51.990273 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:51.990239 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-39.ec2.internal\" not found" Jul 09 03:59:52.071685 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.071639 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal" Jul 09 03:59:52.071763 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.071714 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-39.ec2.internal" Jul 09 03:59:52.090378 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:52.090346 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-39.ec2.internal\" not found" Jul 09 03:59:52.191363 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:52.191326 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-39.ec2.internal\" not found" Jul 09 03:59:52.292425 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:52.292385 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-39.ec2.internal\" not found" Jul 09 03:59:52.328786 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.328755 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 03:59:52.354047 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.354019 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 03:59:52.367176 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.367145 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal" Jul 09 03:59:52.381599 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.381575 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 03:59:52.382610 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.382595 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-39.ec2.internal" Jul 09 03:59:52.383825 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.383812 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 03:59:52.383985 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.383950 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 03:59:52.384035 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.383943 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 03:59:52.384035 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.383957 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 03:59:52.384035 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:52.384009 2576 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://acfa7feb7741b4a30b7de00bedaadd03-e4ea14aa04db216f.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.137.39:49110->100.58.52.99:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-39.ec2.internal" Jul 09 03:59:52.443050 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.442967 2576 apiserver.go:52] "Watching apiserver" Jul 09 03:59:52.463709 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.463678 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 03:59:52.464749 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.464727 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-6mhqx","kube-system/konnectivity-agent-v8v6p","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr","openshift-dns/node-resolver-282rs","openshift-multus/multus-sptvx","openshift-multus/network-metrics-daemon-7zmf5","openshift-network-operator/iptables-alerter-qg99r","openshift-ovn-kubernetes/ovnkube-node-974xm","openshift-cluster-node-tuning-operator/tuned-5wxv6","openshift-image-registry/node-ca-55zxm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal","openshift-multus/multus-additional-cni-plugins-rvsj4","openshift-network-diagnostics/network-check-target-f26vv"] Jul 09 03:59:52.467174 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.467155 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 03:59:52.467425 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.467406 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qg99r" Jul 09 03:59:52.469541 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.469521 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v8v6p" Jul 09 03:59:52.470324 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.470307 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 03:59:52.470324 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.470315 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nssbm\"" Jul 09 03:59:52.470494 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.470479 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 03:59:52.470584 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.470569 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 03:59:52.471780 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.471760 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.472218 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.472201 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zn7lw\"" Jul 09 03:59:52.472279 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.472258 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 03:59:52.472345 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.472205 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 03:59:52.474182 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.474166 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-282rs" Jul 09 03:59:52.474244 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.474184 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6a25c4c5-e4d6-4501-b46c-82a40578cc82-iptables-alerter-script\") pod \"iptables-alerter-qg99r\" (UID: \"6a25c4c5-e4d6-4501-b46c-82a40578cc82\") " pod="openshift-network-operator/iptables-alerter-qg99r" Jul 09 03:59:52.474244 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.474206 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a25c4c5-e4d6-4501-b46c-82a40578cc82-host-slash\") pod \"iptables-alerter-qg99r\" (UID: \"6a25c4c5-e4d6-4501-b46c-82a40578cc82\") " pod="openshift-network-operator/iptables-alerter-qg99r" Jul 09 03:59:52.474244 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.474225 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8tk9\" (UniqueName: \"kubernetes.io/projected/6a25c4c5-e4d6-4501-b46c-82a40578cc82-kube-api-access-s8tk9\") pod \"iptables-alerter-qg99r\" (UID: \"6a25c4c5-e4d6-4501-b46c-82a40578cc82\") " pod="openshift-network-operator/iptables-alerter-qg99r" Jul 09 03:59:52.474404 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.474360 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 03:59:52.474568 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.474551 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 03:59:52.474606 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.474564 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 03:59:52.474802 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.474789 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-85vm9\"" Jul 09 03:59:52.476502 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.476482 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.476938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.476917 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 03:59:52.476938 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.476938 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 03:59:52.477087 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.476970 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f95pl\"" Jul 09 03:59:52.478752 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.478734 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 03:59:52.478925 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:52.478891 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 03:59:52.479139 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.479123 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 03:59:52.479224 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.479139 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 03:59:52.479281 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.479235 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 03:59:52.479438 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.479422 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 03:59:52.479484 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.479470 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-86lh2\"" Jul 09 03:59:52.479974 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.479958 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 03:59:52.481119 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.481106 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 03:59:52.481177 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:52.481162 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 03:59:52.484086 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.484059 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.486615 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.486591 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.488717 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.488686 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 03:54:52 +0000 UTC" deadline="2028-02-15 11:57:45.131357776 +0000 UTC" Jul 09 03:59:52.488717 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.488715 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14071h57m52.642645206s" Jul 09 03:59:52.489098 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.489077 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-55zxm" Jul 09 03:59:52.489493 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.489472 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 03:59:52.489866 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.489851 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 03:59:52.489953 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.489925 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 03:59:52.490022 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.489988 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4lwdd\"" Jul 09 03:59:52.490180 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.489991 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 03:59:52.491216 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.491190 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 03:59:52.493272 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.491644 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 03:59:52.493272 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.491682 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-w6fg4\"" Jul 09 03:59:52.493272 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.491982 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 03:59:52.493272 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.492145 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 03:59:52.493272 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.492249 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rcfwm\"" Jul 09 03:59:52.493272 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.492481 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 03:59:52.493272 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.492829 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 03:59:52.493272 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.492993 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 03:59:52.493670 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.493460 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.495996 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.495975 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 03:59:52.496075 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.496056 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 03:59:52.496339 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.496325 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7l2fp\"" Jul 09 03:59:52.496669 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.496653 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 03:59:52.496762 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:52.496709 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 03:59:52.505417 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.505396 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-smg42" Jul 09 03:59:52.513936 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.513913 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-smg42" Jul 09 03:59:52.567995 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.567976 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 03:59:52.574511 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574488 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-systemd\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.574604 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574520 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-system-cni-dir\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.574604 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574538 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6d52430a-8806-461f-a26a-a4496add6d6f-agent-certs\") pod \"konnectivity-agent-v8v6p\" (UID: \"6d52430a-8806-461f-a26a-a4496add6d6f\") " pod="kube-system/konnectivity-agent-v8v6p" Jul 09 03:59:52.574604 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574553 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-cni-netd\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.574604 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574572 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.574604 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574589 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1aae161d-ab19-425f-a426-685a31ec128d-multus-daemon-config\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.574766 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574632 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prfx6\" (UniqueName: \"kubernetes.io/projected/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-kube-api-access-prfx6\") pod \"network-metrics-daemon-7zmf5\" (UID: \"b13c1cbe-23d9-404e-b028-3efbf70d9fa0\") " pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 03:59:52.574766 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574666 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-log-socket\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.574766 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574693 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.574766 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574710 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/70989833-87c2-4248-a174-c246e7205582-ovnkube-script-lib\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.574766 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574729 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-sysctl-conf\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.574766 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574747 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/953f9350-02c8-4b0b-a2bb-28b305694ddb-host\") pod \"node-ca-55zxm\" (UID: \"953f9350-02c8-4b0b-a2bb-28b305694ddb\") " pod="openshift-image-registry/node-ca-55zxm" Jul 09 03:59:52.574766 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574761 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6d52430a-8806-461f-a26a-a4496add6d6f-konnectivity-ca\") pod \"konnectivity-agent-v8v6p\" (UID: \"6d52430a-8806-461f-a26a-a4496add6d6f\") " pod="kube-system/konnectivity-agent-v8v6p" Jul 09 03:59:52.575049 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574777 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-device-dir\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.575049 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574799 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/68ab2626-7f66-43db-b616-996085b15324-dbus\") pod \"global-pull-secret-syncer-6mhqx\" (UID: \"68ab2626-7f66-43db-b616-996085b15324\") " pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 03:59:52.575049 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574817 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68jdv\" (UniqueName: \"kubernetes.io/projected/70989833-87c2-4248-a174-c246e7205582-kube-api-access-68jdv\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.575049 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574832 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k868m\" (UniqueName: \"kubernetes.io/projected/4f660bde-c939-4cbc-95ba-24dd33bd0165-kube-api-access-k868m\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.575049 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574848 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.575049 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574878 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.575049 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574898 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr79f\" (UniqueName: \"kubernetes.io/projected/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-kube-api-access-hr79f\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.575049 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574930 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6a25c4c5-e4d6-4501-b46c-82a40578cc82-iptables-alerter-script\") pod \"iptables-alerter-qg99r\" (UID: \"6a25c4c5-e4d6-4501-b46c-82a40578cc82\") " pod="openshift-network-operator/iptables-alerter-qg99r" Jul 09 03:59:52.575049 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574951 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef48e4fa-4394-40ac-8960-67646b4c0804-tmp-dir\") pod \"node-resolver-282rs\" (UID: \"ef48e4fa-4394-40ac-8960-67646b4c0804\") " pod="openshift-dns/node-resolver-282rs" Jul 09 03:59:52.575049 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574968 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-hostroot\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.575049 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574983 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-run-netns\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.575049 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.574997 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-run-systemd\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.575049 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575018 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-var-lib-openvswitch\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.575468 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575065 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-run-ovn\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.575468 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575101 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-cni-bin\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.575468 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575130 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-system-cni-dir\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.575468 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575162 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-sysconfig\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.575468 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575190 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-cnibin\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.575468 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575226 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-multus-socket-dir-parent\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.575468 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575243 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-var-lib-cni-bin\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.575468 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575266 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-etc-kubernetes\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.575468 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575280 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-etc-openvswitch\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.575468 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575294 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-var-lib-kubelet\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.575468 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575325 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs\") pod \"network-metrics-daemon-7zmf5\" (UID: \"b13c1cbe-23d9-404e-b028-3efbf70d9fa0\") " pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 03:59:52.575468 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575362 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-run-openvswitch\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.575468 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575384 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-var-lib-kubelet\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.575468 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575409 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-registration-dir\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.576124 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575427 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-sys-fs\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.576124 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575441 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-os-release\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.576124 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575455 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-run-multus-certs\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.576124 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575470 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret\") pod \"global-pull-secret-syncer-6mhqx\" (UID: \"68ab2626-7f66-43db-b616-996085b15324\") " pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 03:59:52.576124 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575484 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-node-log\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.576124 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575499 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/70989833-87c2-4248-a174-c246e7205582-ovn-node-metrics-cert\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.576124 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575516 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-socket-dir\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.576124 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575518 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6a25c4c5-e4d6-4501-b46c-82a40578cc82-iptables-alerter-script\") pod \"iptables-alerter-qg99r\" (UID: \"6a25c4c5-e4d6-4501-b46c-82a40578cc82\") " pod="openshift-network-operator/iptables-alerter-qg99r" Jul 09 03:59:52.576124 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575534 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ftr7\" (UniqueName: \"kubernetes.io/projected/886bda59-684e-43c8-89a0-6a1a394de5b7-kube-api-access-5ftr7\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.576124 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575561 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-var-lib-cni-multus\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.576124 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575586 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gv8j\" (UniqueName: \"kubernetes.io/projected/1aae161d-ab19-425f-a426-685a31ec128d-kube-api-access-9gv8j\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.576124 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575601 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-run\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.576124 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575626 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f660bde-c939-4cbc-95ba-24dd33bd0165-tmp\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.576124 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575677 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/953f9350-02c8-4b0b-a2bb-28b305694ddb-serviceca\") pod \"node-ca-55zxm\" (UID: \"953f9350-02c8-4b0b-a2bb-28b305694ddb\") " pod="openshift-image-registry/node-ca-55zxm" Jul 09 03:59:52.576764 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575711 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-cni-binary-copy\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.576764 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575739 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.576764 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575770 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-systemd-units\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.576764 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575813 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-modprobe-d\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.576764 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575863 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-tuned\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.576764 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575894 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-os-release\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.576764 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575940 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqr47\" (UniqueName: \"kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47\") pod \"network-check-target-f26vv\" (UID: \"e39b46e6-03eb-467c-876a-28c64869086f\") " pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 03:59:52.576764 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575968 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ef48e4fa-4394-40ac-8960-67646b4c0804-hosts-file\") pod \"node-resolver-282rs\" (UID: \"ef48e4fa-4394-40ac-8960-67646b4c0804\") " pod="openshift-dns/node-resolver-282rs" Jul 09 03:59:52.576764 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.575995 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbp84\" (UniqueName: \"kubernetes.io/projected/ef48e4fa-4394-40ac-8960-67646b4c0804-kube-api-access-fbp84\") pod \"node-resolver-282rs\" (UID: \"ef48e4fa-4394-40ac-8960-67646b4c0804\") " pod="openshift-dns/node-resolver-282rs" Jul 09 03:59:52.576764 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576027 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-cnibin\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.576764 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576051 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1aae161d-ab19-425f-a426-685a31ec128d-cni-binary-copy\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.576764 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576074 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-run-k8s-cni-cncf-io\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.576764 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576099 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-kubelet\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.576764 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576123 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-run-ovn-kubernetes\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.577263 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576150 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.577263 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576177 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-kubernetes\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.577263 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576212 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-lib-modules\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.577263 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576238 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-slash\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.577263 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576261 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-sysctl-d\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.577263 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576285 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-run-netns\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.577263 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576309 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-multus-conf-dir\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.577263 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576335 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/68ab2626-7f66-43db-b616-996085b15324-kubelet-config\") pod \"global-pull-secret-syncer-6mhqx\" (UID: \"68ab2626-7f66-43db-b616-996085b15324\") " pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 03:59:52.577263 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576362 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-host\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.577263 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576404 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8tk9\" (UniqueName: \"kubernetes.io/projected/6a25c4c5-e4d6-4501-b46c-82a40578cc82-kube-api-access-s8tk9\") pod \"iptables-alerter-qg99r\" (UID: \"6a25c4c5-e4d6-4501-b46c-82a40578cc82\") " pod="openshift-network-operator/iptables-alerter-qg99r" Jul 09 03:59:52.577263 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576433 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/70989833-87c2-4248-a174-c246e7205582-env-overrides\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.577263 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576456 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-sys\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.577263 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576487 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm6wk\" (UniqueName: \"kubernetes.io/projected/953f9350-02c8-4b0b-a2bb-28b305694ddb-kube-api-access-cm6wk\") pod \"node-ca-55zxm\" (UID: \"953f9350-02c8-4b0b-a2bb-28b305694ddb\") " pod="openshift-image-registry/node-ca-55zxm" Jul 09 03:59:52.577263 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576524 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a25c4c5-e4d6-4501-b46c-82a40578cc82-host-slash\") pod \"iptables-alerter-qg99r\" (UID: \"6a25c4c5-e4d6-4501-b46c-82a40578cc82\") " pod="openshift-network-operator/iptables-alerter-qg99r" Jul 09 03:59:52.577701 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576572 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-multus-cni-dir\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.577701 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576585 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a25c4c5-e4d6-4501-b46c-82a40578cc82-host-slash\") pod \"iptables-alerter-qg99r\" (UID: \"6a25c4c5-e4d6-4501-b46c-82a40578cc82\") " pod="openshift-network-operator/iptables-alerter-qg99r" Jul 09 03:59:52.577701 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.576611 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/70989833-87c2-4248-a174-c246e7205582-ovnkube-config\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.584190 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.584169 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 03:59:52.588024 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.587999 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8tk9\" (UniqueName: \"kubernetes.io/projected/6a25c4c5-e4d6-4501-b46c-82a40578cc82-kube-api-access-s8tk9\") pod \"iptables-alerter-qg99r\" (UID: \"6a25c4c5-e4d6-4501-b46c-82a40578cc82\") " pod="openshift-network-operator/iptables-alerter-qg99r" Jul 09 03:59:52.676852 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.676801 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-multus-cni-dir\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.676852 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.676838 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/70989833-87c2-4248-a174-c246e7205582-ovnkube-config\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.676852 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.676854 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-systemd\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.677152 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.676890 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-system-cni-dir\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.677152 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.676909 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6d52430a-8806-461f-a26a-a4496add6d6f-agent-certs\") pod \"konnectivity-agent-v8v6p\" (UID: \"6d52430a-8806-461f-a26a-a4496add6d6f\") " pod="kube-system/konnectivity-agent-v8v6p" Jul 09 03:59:52.677152 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.676929 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-cni-netd\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.677152 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.676947 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.677152 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677050 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-system-cni-dir\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.677152 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677077 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-systemd\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.677152 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677058 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1aae161d-ab19-425f-a426-685a31ec128d-multus-daemon-config\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.677525 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677159 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prfx6\" (UniqueName: \"kubernetes.io/projected/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-kube-api-access-prfx6\") pod \"network-metrics-daemon-7zmf5\" (UID: \"b13c1cbe-23d9-404e-b028-3efbf70d9fa0\") " pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 03:59:52.677525 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677178 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-multus-cni-dir\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.677525 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677189 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-log-socket\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.677525 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677073 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-etc-selinux\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.677525 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677220 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.677525 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677250 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/70989833-87c2-4248-a174-c246e7205582-ovnkube-script-lib\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.677525 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677278 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-sysctl-conf\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.677525 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677283 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.677525 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677252 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-log-socket\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.677525 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677304 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/953f9350-02c8-4b0b-a2bb-28b305694ddb-host\") pod \"node-ca-55zxm\" (UID: \"953f9350-02c8-4b0b-a2bb-28b305694ddb\") " pod="openshift-image-registry/node-ca-55zxm" Jul 09 03:59:52.677525 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677358 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/953f9350-02c8-4b0b-a2bb-28b305694ddb-host\") pod \"node-ca-55zxm\" (UID: \"953f9350-02c8-4b0b-a2bb-28b305694ddb\") " pod="openshift-image-registry/node-ca-55zxm" Jul 09 03:59:52.677525 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677438 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6d52430a-8806-461f-a26a-a4496add6d6f-konnectivity-ca\") pod \"konnectivity-agent-v8v6p\" (UID: \"6d52430a-8806-461f-a26a-a4496add6d6f\") " pod="kube-system/konnectivity-agent-v8v6p" Jul 09 03:59:52.677525 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677472 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-device-dir\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.677525 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677497 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/68ab2626-7f66-43db-b616-996085b15324-dbus\") pod \"global-pull-secret-syncer-6mhqx\" (UID: \"68ab2626-7f66-43db-b616-996085b15324\") " pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 03:59:52.678295 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677500 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-sysctl-conf\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.678295 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677524 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68jdv\" (UniqueName: \"kubernetes.io/projected/70989833-87c2-4248-a174-c246e7205582-kube-api-access-68jdv\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.678295 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677553 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k868m\" (UniqueName: \"kubernetes.io/projected/4f660bde-c939-4cbc-95ba-24dd33bd0165-kube-api-access-k868m\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.678295 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677558 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-device-dir\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.678295 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677580 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.678295 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677606 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.678295 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677637 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr79f\" (UniqueName: \"kubernetes.io/projected/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-kube-api-access-hr79f\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.678295 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677666 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef48e4fa-4394-40ac-8960-67646b4c0804-tmp-dir\") pod \"node-resolver-282rs\" (UID: \"ef48e4fa-4394-40ac-8960-67646b4c0804\") " pod="openshift-dns/node-resolver-282rs" Jul 09 03:59:52.678295 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677683 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/70989833-87c2-4248-a174-c246e7205582-ovnkube-config\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.678295 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677703 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-hostroot\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.678295 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677746 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-run-netns\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.678295 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677788 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-run-systemd\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.678295 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677793 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/70989833-87c2-4248-a174-c246e7205582-ovnkube-script-lib\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.678295 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677751 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-hostroot\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.679055 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677815 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-var-lib-openvswitch\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.679055 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677849 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-run-ovn\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.679055 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677864 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/68ab2626-7f66-43db-b616-996085b15324-dbus\") pod \"global-pull-secret-syncer-6mhqx\" (UID: \"68ab2626-7f66-43db-b616-996085b15324\") " pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 03:59:52.679055 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677875 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-cni-bin\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.679055 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677891 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-var-lib-openvswitch\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.679055 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677928 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-cni-bin\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.679055 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677930 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-system-cni-dir\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.679055 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677949 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-run-systemd\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.679055 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677969 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-sysconfig\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.679055 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677976 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-system-cni-dir\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.679055 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.677995 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-cnibin\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.679055 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678023 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-multus-socket-dir-parent\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.679055 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678037 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6d52430a-8806-461f-a26a-a4496add6d6f-konnectivity-ca\") pod \"konnectivity-agent-v8v6p\" (UID: \"6d52430a-8806-461f-a26a-a4496add6d6f\") " pod="kube-system/konnectivity-agent-v8v6p" Jul 09 03:59:52.679055 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678051 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-var-lib-cni-bin\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.679055 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678089 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-var-lib-cni-bin\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.679802 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678091 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-etc-kubernetes\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.679802 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678120 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-etc-openvswitch\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.679802 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678124 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-run-netns\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.679802 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678142 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1aae161d-ab19-425f-a426-685a31ec128d-multus-daemon-config\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.679802 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678152 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-var-lib-kubelet\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.679802 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678169 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-etc-openvswitch\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.679802 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678181 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs\") pod \"network-metrics-daemon-7zmf5\" (UID: \"b13c1cbe-23d9-404e-b028-3efbf70d9fa0\") " pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 03:59:52.679802 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678216 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-etc-kubernetes\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.679802 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678227 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-var-lib-kubelet\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.679802 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678262 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-cnibin\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.679802 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678272 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-cni-netd\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.679802 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678305 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-run-openvswitch\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.679802 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678383 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef48e4fa-4394-40ac-8960-67646b4c0804-tmp-dir\") pod \"node-resolver-282rs\" (UID: \"ef48e4fa-4394-40ac-8960-67646b4c0804\") " pod="openshift-dns/node-resolver-282rs" Jul 09 03:59:52.679802 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:52.678393 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:59:52.679802 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678400 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.680538 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678450 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-sysconfig\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.680538 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678453 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-multus-socket-dir-parent\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.680538 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678461 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-var-lib-kubelet\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.680538 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:52.678490 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs podName:b13c1cbe-23d9-404e-b028-3efbf70d9fa0 nodeName:}" failed. No retries permitted until 2026-07-09 03:59:53.17846769 +0000 UTC m=+2.054385284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs") pod "network-metrics-daemon-7zmf5" (UID: "b13c1cbe-23d9-404e-b028-3efbf70d9fa0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:59:52.680538 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678489 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-run-ovn\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.680538 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678559 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-registration-dir\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.680538 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678591 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-sys-fs\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.680538 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678561 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-var-lib-kubelet\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.680538 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678612 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-registration-dir\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.680538 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678635 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.680538 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678648 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-os-release\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.680538 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678656 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-sys-fs\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.680538 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678652 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-run-openvswitch\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.680538 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678697 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-run-multus-certs\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.681076 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678715 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-os-release\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.681076 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678730 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret\") pod \"global-pull-secret-syncer-6mhqx\" (UID: \"68ab2626-7f66-43db-b616-996085b15324\") " pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 03:59:52.681076 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678752 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-run-multus-certs\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.681076 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678764 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-node-log\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.681076 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678791 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/70989833-87c2-4248-a174-c246e7205582-ovn-node-metrics-cert\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.681076 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678814 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-node-log\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.681076 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678817 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-socket-dir\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.681076 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678848 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ftr7\" (UniqueName: \"kubernetes.io/projected/886bda59-684e-43c8-89a0-6a1a394de5b7-kube-api-access-5ftr7\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.681076 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:52.678881 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 03:59:52.681076 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:52.678947 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret podName:68ab2626-7f66-43db-b616-996085b15324 nodeName:}" failed. No retries permitted until 2026-07-09 03:59:53.178930078 +0000 UTC m=+2.054847660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret") pod "global-pull-secret-syncer-6mhqx" (UID: "68ab2626-7f66-43db-b616-996085b15324") : object "kube-system"/"original-pull-secret" not registered Jul 09 03:59:52.681076 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678965 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-socket-dir\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.681076 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.678972 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-var-lib-cni-multus\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.681076 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679008 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gv8j\" (UniqueName: \"kubernetes.io/projected/1aae161d-ab19-425f-a426-685a31ec128d-kube-api-access-9gv8j\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.681076 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679050 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-run\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.681515 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679065 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-var-lib-cni-multus\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.681515 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679071 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 03:59:52.681515 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679074 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f660bde-c939-4cbc-95ba-24dd33bd0165-tmp\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.681515 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679126 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/953f9350-02c8-4b0b-a2bb-28b305694ddb-serviceca\") pod \"node-ca-55zxm\" (UID: \"953f9350-02c8-4b0b-a2bb-28b305694ddb\") " pod="openshift-image-registry/node-ca-55zxm" Jul 09 03:59:52.681515 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679136 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-run\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.681515 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679184 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-cni-binary-copy\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.681515 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679214 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.681515 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679239 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-systemd-units\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.681515 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679264 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-modprobe-d\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.681515 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679286 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-tuned\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.681515 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679313 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-os-release\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.681515 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679317 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/886bda59-684e-43c8-89a0-6a1a394de5b7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.681515 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679361 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqr47\" (UniqueName: \"kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47\") pod \"network-check-target-f26vv\" (UID: \"e39b46e6-03eb-467c-876a-28c64869086f\") " pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 03:59:52.681515 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679405 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ef48e4fa-4394-40ac-8960-67646b4c0804-hosts-file\") pod \"node-resolver-282rs\" (UID: \"ef48e4fa-4394-40ac-8960-67646b4c0804\") " pod="openshift-dns/node-resolver-282rs" Jul 09 03:59:52.681515 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679430 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbp84\" (UniqueName: \"kubernetes.io/projected/ef48e4fa-4394-40ac-8960-67646b4c0804-kube-api-access-fbp84\") pod \"node-resolver-282rs\" (UID: \"ef48e4fa-4394-40ac-8960-67646b4c0804\") " pod="openshift-dns/node-resolver-282rs" Jul 09 03:59:52.682031 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679434 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-os-release\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.682031 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679454 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-cnibin\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.682031 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679491 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1aae161d-ab19-425f-a426-685a31ec128d-cni-binary-copy\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.682031 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679520 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-run-k8s-cni-cncf-io\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.682031 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679566 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-kubelet\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.682031 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679592 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-run-ovn-kubernetes\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.682031 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679618 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.682031 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679646 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-kubernetes\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.682031 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679659 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/953f9350-02c8-4b0b-a2bb-28b305694ddb-serviceca\") pod \"node-ca-55zxm\" (UID: \"953f9350-02c8-4b0b-a2bb-28b305694ddb\") " pod="openshift-image-registry/node-ca-55zxm" Jul 09 03:59:52.682031 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679671 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-lib-modules\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.682031 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679692 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-systemd-units\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.682031 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679720 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-kubelet\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.682031 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679770 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ef48e4fa-4394-40ac-8960-67646b4c0804-hosts-file\") pod \"node-resolver-282rs\" (UID: \"ef48e4fa-4394-40ac-8960-67646b4c0804\") " pod="openshift-dns/node-resolver-282rs" Jul 09 03:59:52.682031 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679785 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-kubernetes\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.682031 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679788 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-cni-binary-copy\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.682513 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679860 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-modprobe-d\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.682513 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679892 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-slash\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.682513 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679949 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-sysctl-d\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.682513 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679958 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-lib-modules\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.682513 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679982 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-run-netns\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.682513 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679998 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-run-k8s-cni-cncf-io\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.682513 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680009 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-multus-conf-dir\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.682513 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680036 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/68ab2626-7f66-43db-b616-996085b15324-kubelet-config\") pod \"global-pull-secret-syncer-6mhqx\" (UID: \"68ab2626-7f66-43db-b616-996085b15324\") " pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 03:59:52.682513 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680043 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-slash\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.682513 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680078 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-sysctl-d\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.682513 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680087 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-host\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.682513 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.679951 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-cnibin\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.682513 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680134 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/70989833-87c2-4248-a174-c246e7205582-env-overrides\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.682513 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680145 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-host-run-netns\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.682513 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680165 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-sys\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.683133 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680191 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm6wk\" (UniqueName: \"kubernetes.io/projected/953f9350-02c8-4b0b-a2bb-28b305694ddb-kube-api-access-cm6wk\") pod \"node-ca-55zxm\" (UID: \"953f9350-02c8-4b0b-a2bb-28b305694ddb\") " pod="openshift-image-registry/node-ca-55zxm" Jul 09 03:59:52.683133 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680200 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1aae161d-ab19-425f-a426-685a31ec128d-multus-conf-dir\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.683133 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680360 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.683133 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680370 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6d52430a-8806-461f-a26a-a4496add6d6f-agent-certs\") pod \"konnectivity-agent-v8v6p\" (UID: \"6d52430a-8806-461f-a26a-a4496add6d6f\") " pod="kube-system/konnectivity-agent-v8v6p" Jul 09 03:59:52.683133 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680416 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70989833-87c2-4248-a174-c246e7205582-host-run-ovn-kubernetes\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.683133 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680437 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/68ab2626-7f66-43db-b616-996085b15324-kubelet-config\") pod \"global-pull-secret-syncer-6mhqx\" (UID: \"68ab2626-7f66-43db-b616-996085b15324\") " pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 03:59:52.683133 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680468 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-host\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.683133 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680543 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1aae161d-ab19-425f-a426-685a31ec128d-cni-binary-copy\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.683133 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680628 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/70989833-87c2-4248-a174-c246e7205582-env-overrides\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.683133 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.680560 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f660bde-c939-4cbc-95ba-24dd33bd0165-sys\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.683133 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.681520 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/70989833-87c2-4248-a174-c246e7205582-ovn-node-metrics-cert\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.683133 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.682711 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4f660bde-c939-4cbc-95ba-24dd33bd0165-etc-tuned\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.683133 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.682765 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f660bde-c939-4cbc-95ba-24dd33bd0165-tmp\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.695711 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:52.695582 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 03:59:52.695711 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:52.695608 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 03:59:52.695711 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:52.695623 2576 projected.go:196] Error preparing data for projected volume kube-api-access-vqr47 for pod openshift-network-diagnostics/network-check-target-f26vv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:59:52.695711 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:52.695697 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47 podName:e39b46e6-03eb-467c-876a-28c64869086f nodeName:}" failed. No retries permitted until 2026-07-09 03:59:53.19567652 +0000 UTC m=+2.071594106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vqr47" (UniqueName: "kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47") pod "network-check-target-f26vv" (UID: "e39b46e6-03eb-467c-876a-28c64869086f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:59:52.696348 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.696324 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prfx6\" (UniqueName: \"kubernetes.io/projected/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-kube-api-access-prfx6\") pod \"network-metrics-daemon-7zmf5\" (UID: \"b13c1cbe-23d9-404e-b028-3efbf70d9fa0\") " pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 03:59:52.697741 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.697716 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm6wk\" (UniqueName: \"kubernetes.io/projected/953f9350-02c8-4b0b-a2bb-28b305694ddb-kube-api-access-cm6wk\") pod \"node-ca-55zxm\" (UID: \"953f9350-02c8-4b0b-a2bb-28b305694ddb\") " pod="openshift-image-registry/node-ca-55zxm" Jul 09 03:59:52.698498 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.698467 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68jdv\" (UniqueName: \"kubernetes.io/projected/70989833-87c2-4248-a174-c246e7205582-kube-api-access-68jdv\") pod \"ovnkube-node-974xm\" (UID: \"70989833-87c2-4248-a174-c246e7205582\") " pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.698692 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.698663 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr79f\" (UniqueName: \"kubernetes.io/projected/a3544bd6-6306-4080-8a5f-d79d2dc9e69b-kube-api-access-hr79f\") pod \"multus-additional-cni-plugins-rvsj4\" (UID: \"a3544bd6-6306-4080-8a5f-d79d2dc9e69b\") " pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.698862 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.698844 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ftr7\" (UniqueName: \"kubernetes.io/projected/886bda59-684e-43c8-89a0-6a1a394de5b7-kube-api-access-5ftr7\") pod \"aws-ebs-csi-driver-node-v8bbr\" (UID: \"886bda59-684e-43c8-89a0-6a1a394de5b7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.699090 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.699070 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbp84\" (UniqueName: \"kubernetes.io/projected/ef48e4fa-4394-40ac-8960-67646b4c0804-kube-api-access-fbp84\") pod \"node-resolver-282rs\" (UID: \"ef48e4fa-4394-40ac-8960-67646b4c0804\") " pod="openshift-dns/node-resolver-282rs" Jul 09 03:59:52.699183 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.699071 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k868m\" (UniqueName: \"kubernetes.io/projected/4f660bde-c939-4cbc-95ba-24dd33bd0165-kube-api-access-k868m\") pod \"tuned-5wxv6\" (UID: \"4f660bde-c939-4cbc-95ba-24dd33bd0165\") " pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.699384 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.699367 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gv8j\" (UniqueName: \"kubernetes.io/projected/1aae161d-ab19-425f-a426-685a31ec128d-kube-api-access-9gv8j\") pod \"multus-sptvx\" (UID: \"1aae161d-ab19-425f-a426-685a31ec128d\") " pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.727694 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:52.727659 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcabda21fc7b0aa2caee43190e64a6bed.slice/crio-c7e6e9843b5dd6664326dcf1bc04311aecdee52ab44108e421f40d3cb1dc8284 WatchSource:0}: Error finding container c7e6e9843b5dd6664326dcf1bc04311aecdee52ab44108e421f40d3cb1dc8284: Status 404 returned error can't find the container with id c7e6e9843b5dd6664326dcf1bc04311aecdee52ab44108e421f40d3cb1dc8284 Jul 09 03:59:52.728420 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:52.728399 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48c34b9b6bc4e55cf67844482ac41f2e.slice/crio-461a105f1d12977aeefac09d5e22bebbf41194fdb8c43b39b2ee16422a8b1939 WatchSource:0}: Error finding container 461a105f1d12977aeefac09d5e22bebbf41194fdb8c43b39b2ee16422a8b1939: Status 404 returned error can't find the container with id 461a105f1d12977aeefac09d5e22bebbf41194fdb8c43b39b2ee16422a8b1939 Jul 09 03:59:52.731932 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.731908 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 03:59:52.798514 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.798480 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qg99r" Jul 09 03:59:52.808466 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:52.808439 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a25c4c5_e4d6_4501_b46c_82a40578cc82.slice/crio-4c63f47ad05944ca0077fc9b5d5752fafc0f7aa1d90e4a23f339469410f17ca6 WatchSource:0}: Error finding container 4c63f47ad05944ca0077fc9b5d5752fafc0f7aa1d90e4a23f339469410f17ca6: Status 404 returned error can't find the container with id 4c63f47ad05944ca0077fc9b5d5752fafc0f7aa1d90e4a23f339469410f17ca6 Jul 09 03:59:52.820088 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.820065 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v8v6p" Jul 09 03:59:52.829203 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:52.829179 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d52430a_8806_461f_a26a_a4496add6d6f.slice/crio-7f9d1e9232b3cd4cab22db4226a0efdf73ac57ba2a6dae0c93952bcc18c6b3f4 WatchSource:0}: Error finding container 7f9d1e9232b3cd4cab22db4226a0efdf73ac57ba2a6dae0c93952bcc18c6b3f4: Status 404 returned error can't find the container with id 7f9d1e9232b3cd4cab22db4226a0efdf73ac57ba2a6dae0c93952bcc18c6b3f4 Jul 09 03:59:52.842521 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.842498 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" Jul 09 03:59:52.850979 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:52.850958 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod886bda59_684e_43c8_89a0_6a1a394de5b7.slice/crio-008d44b61417c851ceb288d2e374cb6d46a71432d33b73181c6d61a88156c5fc WatchSource:0}: Error finding container 008d44b61417c851ceb288d2e374cb6d46a71432d33b73181c6d61a88156c5fc: Status 404 returned error can't find the container with id 008d44b61417c851ceb288d2e374cb6d46a71432d33b73181c6d61a88156c5fc Jul 09 03:59:52.857986 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.857969 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-282rs" Jul 09 03:59:52.864274 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.864256 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sptvx" Jul 09 03:59:52.866168 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:52.866143 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef48e4fa_4394_40ac_8960_67646b4c0804.slice/crio-641a587075fe7101f64fa6538f930c320b82c84fbdd90f6c84c280dbee4aa95e WatchSource:0}: Error finding container 641a587075fe7101f64fa6538f930c320b82c84fbdd90f6c84c280dbee4aa95e: Status 404 returned error can't find the container with id 641a587075fe7101f64fa6538f930c320b82c84fbdd90f6c84c280dbee4aa95e Jul 09 03:59:52.873020 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:52.873000 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aae161d_ab19_425f_a426_685a31ec128d.slice/crio-737785fdd40d16cc14ad530f71bba1e0351ef75b230d68df001e68967903c517 WatchSource:0}: Error finding container 737785fdd40d16cc14ad530f71bba1e0351ef75b230d68df001e68967903c517: Status 404 returned error can't find the container with id 737785fdd40d16cc14ad530f71bba1e0351ef75b230d68df001e68967903c517 Jul 09 03:59:52.884291 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.884266 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 03:59:52.892946 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:52.892925 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70989833_87c2_4248_a174_c246e7205582.slice/crio-45c20586ca24e5850f3c7cb4d76e5e6861f322d989ce9be25c656bf75a061b9c WatchSource:0}: Error finding container 45c20586ca24e5850f3c7cb4d76e5e6861f322d989ce9be25c656bf75a061b9c: Status 404 returned error can't find the container with id 45c20586ca24e5850f3c7cb4d76e5e6861f322d989ce9be25c656bf75a061b9c Jul 09 03:59:52.906979 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.906959 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" Jul 09 03:59:52.917241 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:52.917215 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f660bde_c939_4cbc_95ba_24dd33bd0165.slice/crio-d948d5ebf5f0e82b3ceca07f96c9b9fb0d4cdeb132d61ed5022ca8e05e7f756e WatchSource:0}: Error finding container d948d5ebf5f0e82b3ceca07f96c9b9fb0d4cdeb132d61ed5022ca8e05e7f756e: Status 404 returned error can't find the container with id d948d5ebf5f0e82b3ceca07f96c9b9fb0d4cdeb132d61ed5022ca8e05e7f756e Jul 09 03:59:52.921466 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.921445 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-55zxm" Jul 09 03:59:52.927024 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:52.927006 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rvsj4" Jul 09 03:59:52.930481 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:52.930448 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod953f9350_02c8_4b0b_a2bb_28b305694ddb.slice/crio-fc5b551c642de85322f7e3ee2b8c262e8c3de01d252d04ffa1216b29f6c9a52c WatchSource:0}: Error finding container fc5b551c642de85322f7e3ee2b8c262e8c3de01d252d04ffa1216b29f6c9a52c: Status 404 returned error can't find the container with id fc5b551c642de85322f7e3ee2b8c262e8c3de01d252d04ffa1216b29f6c9a52c Jul 09 03:59:52.937262 ip-10-0-137-39 kubenswrapper[2576]: W0709 03:59:52.937240 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3544bd6_6306_4080_8a5f_d79d2dc9e69b.slice/crio-4b10bde35ca515ed347280f1584ca8716e251256e91b735a346cce765bc0753c WatchSource:0}: Error finding container 4b10bde35ca515ed347280f1584ca8716e251256e91b735a346cce765bc0753c: Status 404 returned error can't find the container with id 4b10bde35ca515ed347280f1584ca8716e251256e91b735a346cce765bc0753c Jul 09 03:59:53.185658 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.185589 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs\") pod \"network-metrics-daemon-7zmf5\" (UID: \"b13c1cbe-23d9-404e-b028-3efbf70d9fa0\") " pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 03:59:53.185925 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.185710 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret\") pod \"global-pull-secret-syncer-6mhqx\" (UID: \"68ab2626-7f66-43db-b616-996085b15324\") " pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 03:59:53.185925 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:53.185735 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:59:53.185925 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:53.185839 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs podName:b13c1cbe-23d9-404e-b028-3efbf70d9fa0 nodeName:}" failed. No retries permitted until 2026-07-09 03:59:54.185810995 +0000 UTC m=+3.061728563 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs") pod "network-metrics-daemon-7zmf5" (UID: "b13c1cbe-23d9-404e-b028-3efbf70d9fa0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:59:53.186166 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:53.185928 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 03:59:53.186166 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:53.186030 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret podName:68ab2626-7f66-43db-b616-996085b15324 nodeName:}" failed. No retries permitted until 2026-07-09 03:59:54.185974383 +0000 UTC m=+3.061891958 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret") pod "global-pull-secret-syncer-6mhqx" (UID: "68ab2626-7f66-43db-b616-996085b15324") : object "kube-system"/"original-pull-secret" not registered Jul 09 03:59:53.226962 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.226931 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 03:59:53.286399 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.286366 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqr47\" (UniqueName: \"kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47\") pod \"network-check-target-f26vv\" (UID: \"e39b46e6-03eb-467c-876a-28c64869086f\") " pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 03:59:53.286572 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:53.286536 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 03:59:53.286617 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:53.286579 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 03:59:53.286617 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:53.286589 2576 projected.go:196] Error preparing data for projected volume kube-api-access-vqr47 for pod openshift-network-diagnostics/network-check-target-f26vv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:59:53.286677 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:53.286648 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47 podName:e39b46e6-03eb-467c-876a-28c64869086f nodeName:}" failed. No retries permitted until 2026-07-09 03:59:54.286628848 +0000 UTC m=+3.162546438 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vqr47" (UniqueName: "kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47") pod "network-check-target-f26vv" (UID: "e39b46e6-03eb-467c-876a-28c64869086f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:59:53.518022 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.517829 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 03:54:53 +0000 UTC" deadline="2027-12-05 05:48:41.43272324 +0000 UTC" Jul 09 03:59:53.518022 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.517901 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12337h48m47.914828189s" Jul 09 03:59:53.621798 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.621154 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 03:59:53.621798 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:53.621286 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 03:59:53.630561 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.630445 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-39.ec2.internal" event={"ID":"cabda21fc7b0aa2caee43190e64a6bed","Type":"ContainerStarted","Data":"c7e6e9843b5dd6664326dcf1bc04311aecdee52ab44108e421f40d3cb1dc8284"} Jul 09 03:59:53.637931 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.637818 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" event={"ID":"4f660bde-c939-4cbc-95ba-24dd33bd0165","Type":"ContainerStarted","Data":"d948d5ebf5f0e82b3ceca07f96c9b9fb0d4cdeb132d61ed5022ca8e05e7f756e"} Jul 09 03:59:53.642944 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.639989 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-282rs" event={"ID":"ef48e4fa-4394-40ac-8960-67646b4c0804","Type":"ContainerStarted","Data":"641a587075fe7101f64fa6538f930c320b82c84fbdd90f6c84c280dbee4aa95e"} Jul 09 03:59:53.646029 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.645944 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v8v6p" event={"ID":"6d52430a-8806-461f-a26a-a4496add6d6f","Type":"ContainerStarted","Data":"7f9d1e9232b3cd4cab22db4226a0efdf73ac57ba2a6dae0c93952bcc18c6b3f4"} Jul 09 03:59:53.647821 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.647744 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qg99r" event={"ID":"6a25c4c5-e4d6-4501-b46c-82a40578cc82","Type":"ContainerStarted","Data":"4c63f47ad05944ca0077fc9b5d5752fafc0f7aa1d90e4a23f339469410f17ca6"} Jul 09 03:59:53.651184 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.651148 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvsj4" event={"ID":"a3544bd6-6306-4080-8a5f-d79d2dc9e69b","Type":"ContainerStarted","Data":"4b10bde35ca515ed347280f1584ca8716e251256e91b735a346cce765bc0753c"} Jul 09 03:59:53.661625 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.661449 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-55zxm" event={"ID":"953f9350-02c8-4b0b-a2bb-28b305694ddb","Type":"ContainerStarted","Data":"fc5b551c642de85322f7e3ee2b8c262e8c3de01d252d04ffa1216b29f6c9a52c"} Jul 09 03:59:53.677369 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.677327 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" event={"ID":"70989833-87c2-4248-a174-c246e7205582","Type":"ContainerStarted","Data":"45c20586ca24e5850f3c7cb4d76e5e6861f322d989ce9be25c656bf75a061b9c"} Jul 09 03:59:53.684222 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.684179 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sptvx" event={"ID":"1aae161d-ab19-425f-a426-685a31ec128d","Type":"ContainerStarted","Data":"737785fdd40d16cc14ad530f71bba1e0351ef75b230d68df001e68967903c517"} Jul 09 03:59:53.691110 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.691049 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" event={"ID":"886bda59-684e-43c8-89a0-6a1a394de5b7","Type":"ContainerStarted","Data":"008d44b61417c851ceb288d2e374cb6d46a71432d33b73181c6d61a88156c5fc"} Jul 09 03:59:53.696610 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:53.696561 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal" event={"ID":"48c34b9b6bc4e55cf67844482ac41f2e","Type":"ContainerStarted","Data":"461a105f1d12977aeefac09d5e22bebbf41194fdb8c43b39b2ee16422a8b1939"} Jul 09 03:59:54.195305 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:54.194450 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs\") pod \"network-metrics-daemon-7zmf5\" (UID: \"b13c1cbe-23d9-404e-b028-3efbf70d9fa0\") " pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 03:59:54.195305 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:54.194514 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret\") pod \"global-pull-secret-syncer-6mhqx\" (UID: \"68ab2626-7f66-43db-b616-996085b15324\") " pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 03:59:54.195305 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:54.194709 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 03:59:54.195305 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:54.194773 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret podName:68ab2626-7f66-43db-b616-996085b15324 nodeName:}" failed. No retries permitted until 2026-07-09 03:59:56.194753991 +0000 UTC m=+5.070671562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret") pod "global-pull-secret-syncer-6mhqx" (UID: "68ab2626-7f66-43db-b616-996085b15324") : object "kube-system"/"original-pull-secret" not registered Jul 09 03:59:54.195305 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:54.195209 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:59:54.195305 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:54.195263 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs podName:b13c1cbe-23d9-404e-b028-3efbf70d9fa0 nodeName:}" failed. No retries permitted until 2026-07-09 03:59:56.195245676 +0000 UTC m=+5.071163249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs") pod "network-metrics-daemon-7zmf5" (UID: "b13c1cbe-23d9-404e-b028-3efbf70d9fa0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:59:54.295390 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:54.295344 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqr47\" (UniqueName: \"kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47\") pod \"network-check-target-f26vv\" (UID: \"e39b46e6-03eb-467c-876a-28c64869086f\") " pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 03:59:54.295586 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:54.295530 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 03:59:54.295586 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:54.295551 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 03:59:54.295586 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:54.295567 2576 projected.go:196] Error preparing data for projected volume kube-api-access-vqr47 for pod openshift-network-diagnostics/network-check-target-f26vv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:59:54.295766 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:54.295634 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47 podName:e39b46e6-03eb-467c-876a-28c64869086f nodeName:}" failed. No retries permitted until 2026-07-09 03:59:56.295614646 +0000 UTC m=+5.171532216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vqr47" (UniqueName: "kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47") pod "network-check-target-f26vv" (UID: "e39b46e6-03eb-467c-876a-28c64869086f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:59:54.518863 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:54.518722 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 03:54:53 +0000 UTC" deadline="2028-02-09 20:20:47.158122845 +0000 UTC" Jul 09 03:59:54.518863 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:54.518768 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13936h20m52.639359889s" Jul 09 03:59:54.619470 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:54.618700 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 03:59:54.619470 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:54.618845 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 03:59:54.619470 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:54.619303 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 03:59:54.619470 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:54.619419 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 03:59:54.725790 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:54.725559 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 03:59:55.556827 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:55.556796 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 03:59:55.620131 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:55.620099 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 03:59:55.620313 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:55.620231 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 03:59:56.214659 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:56.214621 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret\") pod \"global-pull-secret-syncer-6mhqx\" (UID: \"68ab2626-7f66-43db-b616-996085b15324\") " pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 03:59:56.214847 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:56.214744 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs\") pod \"network-metrics-daemon-7zmf5\" (UID: \"b13c1cbe-23d9-404e-b028-3efbf70d9fa0\") " pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 03:59:56.214945 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:56.214902 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:59:56.215001 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:56.214969 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs podName:b13c1cbe-23d9-404e-b028-3efbf70d9fa0 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:00.214948756 +0000 UTC m=+9.090866323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs") pod "network-metrics-daemon-7zmf5" (UID: "b13c1cbe-23d9-404e-b028-3efbf70d9fa0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 03:59:56.215076 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:56.215015 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 03:59:56.215076 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:56.215057 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret podName:68ab2626-7f66-43db-b616-996085b15324 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:00.215044533 +0000 UTC m=+9.090962118 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret") pod "global-pull-secret-syncer-6mhqx" (UID: "68ab2626-7f66-43db-b616-996085b15324") : object "kube-system"/"original-pull-secret" not registered Jul 09 03:59:56.315384 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:56.315340 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqr47\" (UniqueName: \"kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47\") pod \"network-check-target-f26vv\" (UID: \"e39b46e6-03eb-467c-876a-28c64869086f\") " pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 03:59:56.315582 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:56.315553 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 03:59:56.315582 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:56.315580 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 03:59:56.315698 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:56.315596 2576 projected.go:196] Error preparing data for projected volume kube-api-access-vqr47 for pod openshift-network-diagnostics/network-check-target-f26vv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:59:56.315698 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:56.315658 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47 podName:e39b46e6-03eb-467c-876a-28c64869086f nodeName:}" failed. No retries permitted until 2026-07-09 04:00:00.315639513 +0000 UTC m=+9.191557085 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vqr47" (UniqueName: "kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47") pod "network-check-target-f26vv" (UID: "e39b46e6-03eb-467c-876a-28c64869086f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 03:59:56.618354 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:56.617813 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 03:59:56.618354 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:56.617813 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 03:59:56.618354 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:56.617979 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 03:59:56.618354 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:56.618164 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 03:59:57.620653 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:57.620156 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 03:59:57.620653 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:57.620291 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 03:59:58.618256 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:58.618215 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 03:59:58.618452 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:58.618370 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 03:59:58.618566 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:58.618544 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 03:59:58.618714 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:58.618682 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 03:59:59.618599 ip-10-0-137-39 kubenswrapper[2576]: I0709 03:59:59.618444 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 03:59:59.619168 ip-10-0-137-39 kubenswrapper[2576]: E0709 03:59:59.618645 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 04:00:00.253206 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:00.253141 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs\") pod \"network-metrics-daemon-7zmf5\" (UID: \"b13c1cbe-23d9-404e-b028-3efbf70d9fa0\") " pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:00.253206 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:00.253206 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret\") pod \"global-pull-secret-syncer-6mhqx\" (UID: \"68ab2626-7f66-43db-b616-996085b15324\") " pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:00.253516 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:00.253351 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:00:00.253516 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:00.253421 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret podName:68ab2626-7f66-43db-b616-996085b15324 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:08.25339996 +0000 UTC m=+17.129317531 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret") pod "global-pull-secret-syncer-6mhqx" (UID: "68ab2626-7f66-43db-b616-996085b15324") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:00:00.253516 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:00.253347 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:00:00.253516 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:00.253516 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs podName:b13c1cbe-23d9-404e-b028-3efbf70d9fa0 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:08.253493414 +0000 UTC m=+17.129410993 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs") pod "network-metrics-daemon-7zmf5" (UID: "b13c1cbe-23d9-404e-b028-3efbf70d9fa0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:00:00.355155 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:00.354448 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqr47\" (UniqueName: \"kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47\") pod \"network-check-target-f26vv\" (UID: \"e39b46e6-03eb-467c-876a-28c64869086f\") " pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:00.355155 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:00.354663 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:00:00.355155 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:00.354685 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:00:00.355155 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:00.354699 2576 projected.go:196] Error preparing data for projected volume kube-api-access-vqr47 for pod openshift-network-diagnostics/network-check-target-f26vv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:00:00.355155 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:00.354764 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47 podName:e39b46e6-03eb-467c-876a-28c64869086f nodeName:}" failed. No retries permitted until 2026-07-09 04:00:08.354743794 +0000 UTC m=+17.230661363 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vqr47" (UniqueName: "kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47") pod "network-check-target-f26vv" (UID: "e39b46e6-03eb-467c-876a-28c64869086f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:00:00.618712 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:00.618156 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:00.618712 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:00.618210 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:00.618712 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:00.618314 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 04:00:00.618712 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:00.618457 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 04:00:01.618909 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:01.618868 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:01.619411 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:01.619009 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 04:00:02.619560 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:02.618436 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:02.619560 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:02.618579 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 04:00:02.619560 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:02.618990 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:02.619560 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:02.619080 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 04:00:03.618531 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:03.618492 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:03.618715 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:03.618645 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 04:00:04.618923 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:04.618380 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:04.618923 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:04.618418 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:04.618923 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:04.618533 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 04:00:04.618923 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:04.618741 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 04:00:05.617788 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:05.617761 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:05.617961 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:05.617894 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 04:00:06.618191 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:06.618153 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:06.618191 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:06.618185 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:06.618719 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:06.618276 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 04:00:06.618719 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:06.618364 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 04:00:07.618095 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:07.618055 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:07.618279 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:07.618209 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 04:00:08.316805 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:08.316563 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret\") pod \"global-pull-secret-syncer-6mhqx\" (UID: \"68ab2626-7f66-43db-b616-996085b15324\") " pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:08.317013 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:08.316764 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:00:08.317013 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:08.316900 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs\") pod \"network-metrics-daemon-7zmf5\" (UID: \"b13c1cbe-23d9-404e-b028-3efbf70d9fa0\") " pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:08.317013 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:08.316937 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret podName:68ab2626-7f66-43db-b616-996085b15324 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:24.316916324 +0000 UTC m=+33.192833893 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret") pod "global-pull-secret-syncer-6mhqx" (UID: "68ab2626-7f66-43db-b616-996085b15324") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:00:08.317013 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:08.316998 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:00:08.317210 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:08.317049 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs podName:b13c1cbe-23d9-404e-b028-3efbf70d9fa0 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:24.317033088 +0000 UTC m=+33.192950661 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs") pod "network-metrics-daemon-7zmf5" (UID: "b13c1cbe-23d9-404e-b028-3efbf70d9fa0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:00:08.417900 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:08.417863 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqr47\" (UniqueName: \"kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47\") pod \"network-check-target-f26vv\" (UID: \"e39b46e6-03eb-467c-876a-28c64869086f\") " pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:08.418093 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:08.418053 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:00:08.418093 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:08.418079 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:00:08.418093 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:08.418092 2576 projected.go:196] Error preparing data for projected volume kube-api-access-vqr47 for pod openshift-network-diagnostics/network-check-target-f26vv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:00:08.418232 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:08.418159 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47 podName:e39b46e6-03eb-467c-876a-28c64869086f nodeName:}" failed. No retries permitted until 2026-07-09 04:00:24.418141372 +0000 UTC m=+33.294058952 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vqr47" (UniqueName: "kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47") pod "network-check-target-f26vv" (UID: "e39b46e6-03eb-467c-876a-28c64869086f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:00:08.618816 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:08.618726 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:08.619269 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:08.618889 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 04:00:08.619269 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:08.618910 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:08.619269 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:08.619027 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 04:00:09.618120 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:09.618079 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:09.618290 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:09.618212 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 04:00:10.617929 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:10.617892 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:10.618393 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:10.617892 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:10.618393 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:10.618023 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 04:00:10.618393 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:10.618106 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 04:00:11.619203 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:11.619170 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:11.619678 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:11.619284 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 04:00:12.617950 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:12.617910 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:12.618130 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:12.617907 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:12.618130 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:12.618052 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 04:00:12.618216 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:12.618170 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 04:00:13.618751 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:13.618721 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:13.619203 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:13.618872 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 04:00:14.618328 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.618163 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:14.618538 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.618211 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:14.618538 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:14.618415 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 04:00:14.618538 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:14.618463 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 04:00:14.763083 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.763058 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:00:14.763717 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.763371 2576 generic.go:356] "Generic (PLEG): container finished" podID="70989833-87c2-4248-a174-c246e7205582" containerID="45162803d9d010a40e38915a5cbbd03e856cef116eef380f829349749562a358" exitCode=1 Jul 09 04:00:14.763717 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.763443 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" event={"ID":"70989833-87c2-4248-a174-c246e7205582","Type":"ContainerStarted","Data":"d41bff35546d2abba5a407c2f96dad57df53dd73d2690e6f1ebc3b8570e3b4b4"} Jul 09 04:00:14.763717 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.763527 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" event={"ID":"70989833-87c2-4248-a174-c246e7205582","Type":"ContainerStarted","Data":"820c9f502c6e04a364deef012ba6db2ce8aa8000822d818900be7b0380461568"} Jul 09 04:00:14.763717 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.763581 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" event={"ID":"70989833-87c2-4248-a174-c246e7205582","Type":"ContainerStarted","Data":"659f0f7d614d8d96a1f4658442ccd08845f7dab914f6c4b14003528291d3832e"} Jul 09 04:00:14.763717 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.763627 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" event={"ID":"70989833-87c2-4248-a174-c246e7205582","Type":"ContainerStarted","Data":"c82855c406a031acbec9781037511830fb270e48d91a3962f734cb89363c72ad"} Jul 09 04:00:14.763717 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.763662 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" event={"ID":"70989833-87c2-4248-a174-c246e7205582","Type":"ContainerDied","Data":"45162803d9d010a40e38915a5cbbd03e856cef116eef380f829349749562a358"} Jul 09 04:00:14.763717 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.763705 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" event={"ID":"70989833-87c2-4248-a174-c246e7205582","Type":"ContainerStarted","Data":"ae88d69a9a6480adda77c5e8d28ee112dcb5f94669806ebea78beda96987a7b0"} Jul 09 04:00:14.764816 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.764796 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sptvx" event={"ID":"1aae161d-ab19-425f-a426-685a31ec128d","Type":"ContainerStarted","Data":"211f4d8a68c323836fed7d591ea3b7dd99e2f572d0fbdf5ae52b7bda1170ab8b"} Jul 09 04:00:14.766159 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.766137 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-39.ec2.internal" event={"ID":"cabda21fc7b0aa2caee43190e64a6bed","Type":"ContainerStarted","Data":"eb733e16e0f4016e1f3ab9124e896a47a0e316708f366c21ee14b9767c57a631"} Jul 09 04:00:14.766266 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.766254 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-39.ec2.internal" Jul 09 04:00:14.767406 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.767388 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" event={"ID":"4f660bde-c939-4cbc-95ba-24dd33bd0165","Type":"ContainerStarted","Data":"eacec9cd2f52e9d07f1a28d5e735e85c4307672f42d3c0c9acdb0667dd3bd252"} Jul 09 04:00:14.782110 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.782092 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 04:00:14.782747 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.782729 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-137-39.ec2.internal"] Jul 09 04:00:14.815505 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.815418 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-sptvx" podStartSLOduration=1.960413402 podStartE2EDuration="22.815403321s" totalImagesPullingTime="20.854989919s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:14.799583978 +0000 UTC m=+23.675501566" watchObservedRunningTime="2026-07-09 04:00:14.815403321 +0000 UTC m=+23.691320909" Jul 09 04:00:14.815945 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.815918 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-39.ec2.internal" podStartSLOduration=0.815910257 podStartE2EDuration="815.910257ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:00:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:14.815278082 +0000 UTC m=+23.691195669" watchObservedRunningTime="2026-07-09 04:00:14.815910257 +0000 UTC m=+23.691827845" Jul 09 04:00:14.833145 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:14.833105 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5wxv6" podStartSLOduration=1.9988963210000001 podStartE2EDuration="22.83309293s" totalImagesPullingTime="20.834196609s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:14.832378397 +0000 UTC m=+23.708295986" watchObservedRunningTime="2026-07-09 04:00:14.83309293 +0000 UTC m=+23.709010519" Jul 09 04:00:15.618972 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:15.618695 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:15.618972 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:15.618966 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 04:00:15.770775 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:15.770739 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-282rs" event={"ID":"ef48e4fa-4394-40ac-8960-67646b4c0804","Type":"ContainerStarted","Data":"6f099017b31cea198bc2e45c0d6bcaf49e583f49fc92a2f19b844b52be1f2868"} Jul 09 04:00:15.772269 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:15.772246 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v8v6p" event={"ID":"6d52430a-8806-461f-a26a-a4496add6d6f","Type":"ContainerStarted","Data":"9d65432291a1642c72022462a986b788e7afae997385d51eb7f77a896b8127e6"} Jul 09 04:00:15.773707 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:15.773685 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qg99r" event={"ID":"6a25c4c5-e4d6-4501-b46c-82a40578cc82","Type":"ContainerStarted","Data":"87c51905cd8684bc42c12fc82b1a6222744eb1e5c2d0a6a5b95e8d5d7aa1dc71"} Jul 09 04:00:15.775275 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:15.775256 2576 generic.go:356] "Generic (PLEG): container finished" podID="a3544bd6-6306-4080-8a5f-d79d2dc9e69b" containerID="fa06633d9135acaa7d01a8797e08ebb305cef7b596593970e3b2d088ac2af10a" exitCode=0 Jul 09 04:00:15.775344 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:15.775319 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvsj4" event={"ID":"a3544bd6-6306-4080-8a5f-d79d2dc9e69b","Type":"ContainerDied","Data":"fa06633d9135acaa7d01a8797e08ebb305cef7b596593970e3b2d088ac2af10a"} Jul 09 04:00:15.776913 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:15.776894 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-55zxm" event={"ID":"953f9350-02c8-4b0b-a2bb-28b305694ddb","Type":"ContainerStarted","Data":"eaeed2bce7f5e24e068159bf79ace432513722e4f6c218adaf75fded67924c03"} Jul 09 04:00:15.778449 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:15.778424 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" event={"ID":"886bda59-684e-43c8-89a0-6a1a394de5b7","Type":"ContainerStarted","Data":"8b40b6cc185b44e395b75db1d5bfc0fcc8a5491c761846502e07efa10e9187b0"} Jul 09 04:00:15.779724 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:15.779705 2576 generic.go:356] "Generic (PLEG): container finished" podID="48c34b9b6bc4e55cf67844482ac41f2e" containerID="1b6bd55f713b79bc4a0f7ad6696625c4923dab9078975481abbca3c45f194565" exitCode=0 Jul 09 04:00:15.779830 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:15.779799 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal" event={"ID":"48c34b9b6bc4e55cf67844482ac41f2e","Type":"ContainerDied","Data":"1b6bd55f713b79bc4a0f7ad6696625c4923dab9078975481abbca3c45f194565"} Jul 09 04:00:15.780260 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:15.780228 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-39.ec2.internal" Jul 09 04:00:15.787687 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:15.787650 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-282rs" podStartSLOduration=2.902624549 podStartE2EDuration="23.787637581s" totalImagesPullingTime="20.885013032s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:15.787158956 +0000 UTC m=+24.663076543" watchObservedRunningTime="2026-07-09 04:00:15.787637581 +0000 UTC m=+24.663555168" Jul 09 04:00:15.790882 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:15.790866 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 04:00:15.790950 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:15.790903 2576 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-137-39.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-39.ec2.internal" Jul 09 04:00:15.801645 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:15.801608 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-55zxm" podStartSLOduration=3.06413956 podStartE2EDuration="23.80159515s" totalImagesPullingTime="20.73745559s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:15.801415309 +0000 UTC m=+24.677332897" watchObservedRunningTime="2026-07-09 04:00:15.80159515 +0000 UTC m=+24.677512739" Jul 09 04:00:15.832780 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:15.832719 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qg99r" podStartSLOduration=2.9132775 podStartE2EDuration="23.832699966s" totalImagesPullingTime="20.919422466s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:15.832537783 +0000 UTC m=+24.708455372" watchObservedRunningTime="2026-07-09 04:00:15.832699966 +0000 UTC m=+24.708617568" Jul 09 04:00:15.849167 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:15.849112 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-v8v6p" podStartSLOduration=2.9504075849999998 podStartE2EDuration="23.849094806s" totalImagesPullingTime="20.898687221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:15.848802952 +0000 UTC m=+24.724720541" watchObservedRunningTime="2026-07-09 04:00:15.849094806 +0000 UTC m=+24.725012395" Jul 09 04:00:16.617913 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:16.617881 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:16.617913 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:16.617911 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:16.618210 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:16.618025 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 04:00:16.618210 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:16.618095 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 04:00:16.785808 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:16.785786 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:00:16.786224 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:16.786194 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" event={"ID":"70989833-87c2-4248-a174-c246e7205582","Type":"ContainerStarted","Data":"9256b04177430153a1a93d1f01df03277a890fc914f11526cf5226322d06772c"} Jul 09 04:00:16.787824 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:16.787800 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal" event={"ID":"48c34b9b6bc4e55cf67844482ac41f2e","Type":"ContainerStarted","Data":"0decf5e73aeda95991cbb8cf5544b62f74113af4505e2d1a00f2ff7053e1e757"} Jul 09 04:00:16.803316 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:16.803268 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-39.ec2.internal" podStartSLOduration=23.80325476 podStartE2EDuration="23.80325476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:59:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:16.802919284 +0000 UTC m=+25.678836873" watchObservedRunningTime="2026-07-09 04:00:16.80325476 +0000 UTC m=+25.679172357" Jul 09 04:00:17.077834 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:17.077794 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 04:00:17.568268 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:17.568163 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T04:00:17.077820881Z","UUID":"e7de7949-5a41-4b2d-9444-9465794bb54d","Handler":null,"Name":"","Endpoint":""} Jul 09 04:00:17.569793 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:17.569776 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 04:00:17.569898 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:17.569810 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 04:00:17.618482 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:17.618451 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:17.618647 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:17.618563 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 04:00:17.791420 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:17.791383 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" event={"ID":"886bda59-684e-43c8-89a0-6a1a394de5b7","Type":"ContainerStarted","Data":"d19e2ca248f2ef682e8350f11c900567232b264709a7fe14782ce9dc97167b18"} Jul 09 04:00:18.618034 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:18.618001 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:18.618034 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:18.618020 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:18.618282 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:18.618127 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 04:00:18.618282 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:18.618237 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 04:00:19.069419 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:19.069171 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-v8v6p" Jul 09 04:00:19.070080 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:19.069805 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-v8v6p" Jul 09 04:00:19.617895 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:19.617805 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:19.618098 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:19.617989 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 04:00:19.803923 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:19.803893 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:00:19.804397 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:19.804344 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" event={"ID":"70989833-87c2-4248-a174-c246e7205582","Type":"ContainerStarted","Data":"5ce31eb6c65bb0a518c23f2fc68c4c3080ddee120427d6fa3391b8dcc882b6c7"} Jul 09 04:00:19.804619 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:19.804588 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 04:00:19.804724 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:19.804652 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 04:00:19.804805 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:19.804792 2576 scope.go:117] "RemoveContainer" containerID="45162803d9d010a40e38915a5cbbd03e856cef116eef380f829349749562a358" Jul 09 04:00:19.806897 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:19.806867 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" event={"ID":"886bda59-684e-43c8-89a0-6a1a394de5b7","Type":"ContainerStarted","Data":"7b40ad1f45f7fa7d5d59ebb1f9bc4cf5a7c24f197658830259770e5d7b87c388"} Jul 09 04:00:19.807245 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:19.807222 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-v8v6p" Jul 09 04:00:19.808288 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:19.808269 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-v8v6p" Jul 09 04:00:19.823080 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:19.823063 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 04:00:19.887707 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:19.887661 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v8bbr" podStartSLOduration=3.450221983 podStartE2EDuration="27.887647695s" totalImagesPullingTime="24.437425712s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:59:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:19.88755266 +0000 UTC m=+28.763470248" watchObservedRunningTime="2026-07-09 04:00:19.887647695 +0000 UTC m=+28.763565283" Jul 09 04:00:20.618588 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:20.618556 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:20.619387 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:20.618705 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 04:00:20.619387 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:20.618743 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:20.619387 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:20.618879 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 04:00:20.813882 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:20.813834 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:00:20.814264 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:20.814234 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" event={"ID":"70989833-87c2-4248-a174-c246e7205582","Type":"ContainerStarted","Data":"7ebabcd83f5aaa2735b04c9e86ba7cbb0b641bc013b38b81b2ddbb43f5e7225c"} Jul 09 04:00:20.814499 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:20.814483 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 04:00:20.830907 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:20.830880 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 04:00:20.846669 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:20.845068 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" podStartSLOduration=7.725762938 podStartE2EDuration="28.845047625s" totalImagesPullingTime="21.119284687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:20.842153303 +0000 UTC m=+29.718070891" watchObservedRunningTime="2026-07-09 04:00:20.845047625 +0000 UTC m=+29.720965214" Jul 09 04:00:20.867210 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:20.867174 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7zmf5"] Jul 09 04:00:20.867355 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:20.867302 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:20.867430 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:20.867388 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 04:00:20.869920 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:20.869864 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f26vv"] Jul 09 04:00:20.870035 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:20.869949 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:20.870106 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:20.870030 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 04:00:20.878963 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:20.878937 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6mhqx"] Jul 09 04:00:20.879089 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:20.879037 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:20.879152 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:20.879134 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 04:00:22.619349 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:22.618455 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:22.619349 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:22.618808 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 04:00:22.619349 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:22.618534 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:22.619349 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:22.618923 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 04:00:22.619349 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:22.618503 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:22.619349 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:22.619007 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 04:00:22.820063 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:22.820029 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvsj4" event={"ID":"a3544bd6-6306-4080-8a5f-d79d2dc9e69b","Type":"ContainerStarted","Data":"3d4b35a6e8b227f70ebd34813716bd085aa8e2c3716122cae2be3682780c9225"} Jul 09 04:00:23.823883 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:23.823779 2576 generic.go:356] "Generic (PLEG): container finished" podID="a3544bd6-6306-4080-8a5f-d79d2dc9e69b" containerID="3d4b35a6e8b227f70ebd34813716bd085aa8e2c3716122cae2be3682780c9225" exitCode=0 Jul 09 04:00:23.823883 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:23.823876 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvsj4" event={"ID":"a3544bd6-6306-4080-8a5f-d79d2dc9e69b","Type":"ContainerDied","Data":"3d4b35a6e8b227f70ebd34813716bd085aa8e2c3716122cae2be3682780c9225"} Jul 09 04:00:24.342612 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:24.342579 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs\") pod \"network-metrics-daemon-7zmf5\" (UID: \"b13c1cbe-23d9-404e-b028-3efbf70d9fa0\") " pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:24.342612 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:24.342615 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret\") pod \"global-pull-secret-syncer-6mhqx\" (UID: \"68ab2626-7f66-43db-b616-996085b15324\") " pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:24.342788 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:24.342723 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:00:24.342788 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:24.342775 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret podName:68ab2626-7f66-43db-b616-996085b15324 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:56.342761652 +0000 UTC m=+65.218679218 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret") pod "global-pull-secret-syncer-6mhqx" (UID: "68ab2626-7f66-43db-b616-996085b15324") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:00:24.342901 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:24.342724 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:00:24.342901 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:24.342878 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs podName:b13c1cbe-23d9-404e-b028-3efbf70d9fa0 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:56.342859876 +0000 UTC m=+65.218777460 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs") pod "network-metrics-daemon-7zmf5" (UID: "b13c1cbe-23d9-404e-b028-3efbf70d9fa0") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:00:24.443814 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:24.443720 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqr47\" (UniqueName: \"kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47\") pod \"network-check-target-f26vv\" (UID: \"e39b46e6-03eb-467c-876a-28c64869086f\") " pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:24.443991 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:24.443900 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:00:24.443991 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:24.443926 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:00:24.443991 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:24.443940 2576 projected.go:196] Error preparing data for projected volume kube-api-access-vqr47 for pod openshift-network-diagnostics/network-check-target-f26vv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:00:24.444130 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:24.444005 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47 podName:e39b46e6-03eb-467c-876a-28c64869086f nodeName:}" failed. No retries permitted until 2026-07-09 04:00:56.443984363 +0000 UTC m=+65.319901953 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vqr47" (UniqueName: "kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47") pod "network-check-target-f26vv" (UID: "e39b46e6-03eb-467c-876a-28c64869086f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:00:24.617792 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:24.617763 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:24.617992 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:24.617806 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:24.617992 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:24.617760 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:24.617992 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:24.617915 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 04:00:24.618145 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:24.617998 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 04:00:24.618145 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:24.618067 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 04:00:25.830866 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:25.830817 2576 generic.go:356] "Generic (PLEG): container finished" podID="a3544bd6-6306-4080-8a5f-d79d2dc9e69b" containerID="1ef42bd368738ed99e16469b5fe879621b7886aec869f0ce2b5a4eba2403918e" exitCode=0 Jul 09 04:00:25.831233 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:25.830873 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvsj4" event={"ID":"a3544bd6-6306-4080-8a5f-d79d2dc9e69b","Type":"ContainerDied","Data":"1ef42bd368738ed99e16469b5fe879621b7886aec869f0ce2b5a4eba2403918e"} Jul 09 04:00:26.617785 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:26.617749 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:26.617955 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:26.617808 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:26.617955 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:26.617820 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:26.617955 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:26.617934 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f26vv" podUID="e39b46e6-03eb-467c-876a-28c64869086f" Jul 09 04:00:26.618071 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:26.617977 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6mhqx" podUID="68ab2626-7f66-43db-b616-996085b15324" Jul 09 04:00:26.618071 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:26.618043 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7zmf5" podUID="b13c1cbe-23d9-404e-b028-3efbf70d9fa0" Jul 09 04:00:26.960720 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:26.960692 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-39.ec2.internal" event="NodeReady" Jul 09 04:00:26.961104 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:26.960834 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 04:00:26.994166 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:26.994139 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-bfbcddb55-fnvw7"] Jul 09 04:00:27.006965 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.006937 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.007410 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.007386 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bfbcddb55-fnvw7"] Jul 09 04:00:27.009711 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.009687 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gcm68"] Jul 09 04:00:27.010596 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.010380 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 04:00:27.010596 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.010515 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 04:00:27.010773 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.010755 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-brrqc\"" Jul 09 04:00:27.011011 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.010995 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 04:00:27.023833 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.023812 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 04:00:27.029074 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.029057 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p7z6b"] Jul 09 04:00:27.029191 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.029175 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:27.032666 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.032651 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 04:00:27.033983 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.033963 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 04:00:27.034714 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.034695 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wgmf2\"" Jul 09 04:00:27.047614 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.047592 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gcm68"] Jul 09 04:00:27.047714 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.047622 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p7z6b"] Jul 09 04:00:27.047714 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.047701 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p7z6b" Jul 09 04:00:27.051762 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.051734 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 04:00:27.051762 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.051748 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dfx4h\"" Jul 09 04:00:27.051922 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.051827 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 04:00:27.052211 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.052194 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 04:00:27.165352 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.165320 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53a12a15-f2c0-44ef-a267-6a540e093a33-cert\") pod \"ingress-canary-p7z6b\" (UID: \"53a12a15-f2c0-44ef-a267-6a540e093a33\") " pod="openshift-ingress-canary/ingress-canary-p7z6b" Jul 09 04:00:27.165352 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.165355 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmdpn\" (UniqueName: \"kubernetes.io/projected/53a12a15-f2c0-44ef-a267-6a540e093a33-kube-api-access-gmdpn\") pod \"ingress-canary-p7z6b\" (UID: \"53a12a15-f2c0-44ef-a267-6a540e093a33\") " pod="openshift-ingress-canary/ingress-canary-p7z6b" Jul 09 04:00:27.165536 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.165379 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:27.165536 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.165468 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d595e0d2-6955-4f99-9531-47b6c1ce95a3-config-volume\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:27.165536 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.165504 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93ea36c7-0908-4ab9-8b33-fd167f9497ec-trusted-ca\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.165536 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.165527 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93ea36c7-0908-4ab9-8b33-fd167f9497ec-installation-pull-secrets\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.165653 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.165554 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thndc\" (UniqueName: \"kubernetes.io/projected/d595e0d2-6955-4f99-9531-47b6c1ce95a3-kube-api-access-thndc\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:27.165653 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.165571 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.165653 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.165590 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d595e0d2-6955-4f99-9531-47b6c1ce95a3-tmp-dir\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:27.165653 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.165627 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93ea36c7-0908-4ab9-8b33-fd167f9497ec-ca-trust-extracted\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.165653 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.165648 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j65tc\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-kube-api-access-j65tc\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.165794 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.165669 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-certificates\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.165794 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.165703 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93ea36c7-0908-4ab9-8b33-fd167f9497ec-image-registry-private-configuration\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.165794 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.165724 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-bound-sa-token\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.266365 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.266288 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93ea36c7-0908-4ab9-8b33-fd167f9497ec-trusted-ca\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.266365 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.266328 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93ea36c7-0908-4ab9-8b33-fd167f9497ec-installation-pull-secrets\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.266365 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.266352 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thndc\" (UniqueName: \"kubernetes.io/projected/d595e0d2-6955-4f99-9531-47b6c1ce95a3-kube-api-access-thndc\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:27.266614 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.266476 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.266614 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.266514 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d595e0d2-6955-4f99-9531-47b6c1ce95a3-tmp-dir\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:27.266614 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.266544 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93ea36c7-0908-4ab9-8b33-fd167f9497ec-ca-trust-extracted\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.266614 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:27.266601 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:00:27.266761 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:27.266618 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bfbcddb55-fnvw7: secret "image-registry-tls" not found Jul 09 04:00:27.266761 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.266660 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j65tc\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-kube-api-access-j65tc\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.266761 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:27.266675 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls podName:93ea36c7-0908-4ab9-8b33-fd167f9497ec nodeName:}" failed. No retries permitted until 2026-07-09 04:00:27.766653574 +0000 UTC m=+36.642571151 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls") pod "image-registry-bfbcddb55-fnvw7" (UID: "93ea36c7-0908-4ab9-8b33-fd167f9497ec") : secret "image-registry-tls" not found Jul 09 04:00:27.266761 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.266715 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-certificates\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.266963 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.266771 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93ea36c7-0908-4ab9-8b33-fd167f9497ec-image-registry-private-configuration\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.266963 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.266794 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-bound-sa-token\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.266963 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.266815 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53a12a15-f2c0-44ef-a267-6a540e093a33-cert\") pod \"ingress-canary-p7z6b\" (UID: \"53a12a15-f2c0-44ef-a267-6a540e093a33\") " pod="openshift-ingress-canary/ingress-canary-p7z6b" Jul 09 04:00:27.266963 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.266856 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmdpn\" (UniqueName: \"kubernetes.io/projected/53a12a15-f2c0-44ef-a267-6a540e093a33-kube-api-access-gmdpn\") pod \"ingress-canary-p7z6b\" (UID: \"53a12a15-f2c0-44ef-a267-6a540e093a33\") " pod="openshift-ingress-canary/ingress-canary-p7z6b" Jul 09 04:00:27.266963 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.266878 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:27.267148 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.266963 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d595e0d2-6955-4f99-9531-47b6c1ce95a3-config-volume\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:27.267148 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.267021 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d595e0d2-6955-4f99-9531-47b6c1ce95a3-tmp-dir\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:27.267148 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.267105 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93ea36c7-0908-4ab9-8b33-fd167f9497ec-ca-trust-extracted\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.267268 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:27.267171 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:00:27.267268 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:27.267204 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls podName:d595e0d2-6955-4f99-9531-47b6c1ce95a3 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:27.767192483 +0000 UTC m=+36.643110050 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls") pod "dns-default-gcm68" (UID: "d595e0d2-6955-4f99-9531-47b6c1ce95a3") : secret "dns-default-metrics-tls" not found Jul 09 04:00:27.267533 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.267354 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-certificates\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.268111 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.268087 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93ea36c7-0908-4ab9-8b33-fd167f9497ec-trusted-ca\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.270552 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.270526 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93ea36c7-0908-4ab9-8b33-fd167f9497ec-image-registry-private-configuration\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.270619 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.270549 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/53a12a15-f2c0-44ef-a267-6a540e093a33-cert\") pod \"ingress-canary-p7z6b\" (UID: \"53a12a15-f2c0-44ef-a267-6a540e093a33\") " pod="openshift-ingress-canary/ingress-canary-p7z6b" Jul 09 04:00:27.270619 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.270581 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93ea36c7-0908-4ab9-8b33-fd167f9497ec-installation-pull-secrets\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.277911 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.277889 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmdpn\" (UniqueName: \"kubernetes.io/projected/53a12a15-f2c0-44ef-a267-6a540e093a33-kube-api-access-gmdpn\") pod \"ingress-canary-p7z6b\" (UID: \"53a12a15-f2c0-44ef-a267-6a540e093a33\") " pod="openshift-ingress-canary/ingress-canary-p7z6b" Jul 09 04:00:27.277995 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.277898 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j65tc\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-kube-api-access-j65tc\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.278049 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.278029 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-bound-sa-token\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.279352 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.279333 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d595e0d2-6955-4f99-9531-47b6c1ce95a3-config-volume\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:27.280775 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.280757 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thndc\" (UniqueName: \"kubernetes.io/projected/d595e0d2-6955-4f99-9531-47b6c1ce95a3-kube-api-access-thndc\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:27.355061 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.355021 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p7z6b" Jul 09 04:00:27.534303 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.534274 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p7z6b"] Jul 09 04:00:27.538883 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:00:27.538852 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53a12a15_f2c0_44ef_a267_6a540e093a33.slice/crio-1221db4b7caae93138f9860cdf64100fd8e8cfab1080dab809838eeaa097712a WatchSource:0}: Error finding container 1221db4b7caae93138f9860cdf64100fd8e8cfab1080dab809838eeaa097712a: Status 404 returned error can't find the container with id 1221db4b7caae93138f9860cdf64100fd8e8cfab1080dab809838eeaa097712a Jul 09 04:00:27.770865 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.770755 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:27.770865 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.770831 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:27.771063 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:27.770930 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:00:27.771063 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:27.770951 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:00:27.771063 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:27.770961 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bfbcddb55-fnvw7: secret "image-registry-tls" not found Jul 09 04:00:27.771063 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:27.770993 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls podName:d595e0d2-6955-4f99-9531-47b6c1ce95a3 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:28.770977245 +0000 UTC m=+37.646894815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls") pod "dns-default-gcm68" (UID: "d595e0d2-6955-4f99-9531-47b6c1ce95a3") : secret "dns-default-metrics-tls" not found Jul 09 04:00:27.771063 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:27.771009 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls podName:93ea36c7-0908-4ab9-8b33-fd167f9497ec nodeName:}" failed. No retries permitted until 2026-07-09 04:00:28.770998604 +0000 UTC m=+37.646916171 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls") pod "image-registry-bfbcddb55-fnvw7" (UID: "93ea36c7-0908-4ab9-8b33-fd167f9497ec") : secret "image-registry-tls" not found Jul 09 04:00:27.838291 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.838259 2576 generic.go:356] "Generic (PLEG): container finished" podID="a3544bd6-6306-4080-8a5f-d79d2dc9e69b" containerID="9308dd0d6547805834885c290e3f6625f69f4c2c64949cd6997be4c14382b181" exitCode=0 Jul 09 04:00:27.838463 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.838333 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvsj4" event={"ID":"a3544bd6-6306-4080-8a5f-d79d2dc9e69b","Type":"ContainerDied","Data":"9308dd0d6547805834885c290e3f6625f69f4c2c64949cd6997be4c14382b181"} Jul 09 04:00:27.839498 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:27.839465 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p7z6b" event={"ID":"53a12a15-f2c0-44ef-a267-6a540e093a33","Type":"ContainerStarted","Data":"1221db4b7caae93138f9860cdf64100fd8e8cfab1080dab809838eeaa097712a"} Jul 09 04:00:28.617974 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:28.617938 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:28.617974 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:28.617963 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:28.618815 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:28.617938 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:28.620887 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:28.620864 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 04:00:28.621028 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:28.620900 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 04:00:28.622234 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:28.622116 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 04:00:28.622234 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:28.622128 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-25smd\"" Jul 09 04:00:28.622234 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:28.622146 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fn82r\"" Jul 09 04:00:28.622234 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:28.622128 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 04:00:28.780220 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:28.780188 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:28.780380 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:28.780282 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:28.780380 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:28.780358 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:00:28.780500 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:28.780429 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls podName:d595e0d2-6955-4f99-9531-47b6c1ce95a3 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:30.780409958 +0000 UTC m=+39.656327543 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls") pod "dns-default-gcm68" (UID: "d595e0d2-6955-4f99-9531-47b6c1ce95a3") : secret "dns-default-metrics-tls" not found Jul 09 04:00:28.780500 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:28.780440 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:00:28.780500 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:28.780458 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bfbcddb55-fnvw7: secret "image-registry-tls" not found Jul 09 04:00:28.780650 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:28.780514 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls podName:93ea36c7-0908-4ab9-8b33-fd167f9497ec nodeName:}" failed. No retries permitted until 2026-07-09 04:00:30.7804964 +0000 UTC m=+39.656413970 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls") pod "image-registry-bfbcddb55-fnvw7" (UID: "93ea36c7-0908-4ab9-8b33-fd167f9497ec") : secret "image-registry-tls" not found Jul 09 04:00:30.797273 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:30.797029 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:30.797687 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:30.797337 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:30.797687 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:30.797198 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:00:30.797687 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:30.797558 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bfbcddb55-fnvw7: secret "image-registry-tls" not found Jul 09 04:00:30.797687 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:30.797625 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls podName:93ea36c7-0908-4ab9-8b33-fd167f9497ec nodeName:}" failed. No retries permitted until 2026-07-09 04:00:34.797603607 +0000 UTC m=+43.673521197 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls") pod "image-registry-bfbcddb55-fnvw7" (UID: "93ea36c7-0908-4ab9-8b33-fd167f9497ec") : secret "image-registry-tls" not found Jul 09 04:00:30.798036 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:30.797539 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:00:30.798036 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:30.798012 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls podName:d595e0d2-6955-4f99-9531-47b6c1ce95a3 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:34.797992546 +0000 UTC m=+43.673910131 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls") pod "dns-default-gcm68" (UID: "d595e0d2-6955-4f99-9531-47b6c1ce95a3") : secret "dns-default-metrics-tls" not found Jul 09 04:00:30.850291 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:30.850221 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p7z6b" event={"ID":"53a12a15-f2c0-44ef-a267-6a540e093a33","Type":"ContainerStarted","Data":"2924db065dc78329f8d40a22825e511b0a024160974f4beecd886cfdc70b8243"} Jul 09 04:00:30.866964 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:30.866901 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p7z6b" podStartSLOduration=1.812284537 podStartE2EDuration="4.866878628s" totalImagesPullingTime="3.054594091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:30.86638043 +0000 UTC m=+39.742298018" watchObservedRunningTime="2026-07-09 04:00:30.866878628 +0000 UTC m=+39.742796218" Jul 09 04:00:33.861761 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:33.861722 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvsj4" event={"ID":"a3544bd6-6306-4080-8a5f-d79d2dc9e69b","Type":"ContainerStarted","Data":"009e09e5382bba199eb8bd9d05248511f9e4244dc6c8b00be3ebc7d2b937d728"} Jul 09 04:00:34.827800 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:34.827763 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:34.827983 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:34.827818 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:34.827983 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:34.827960 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:00:34.827983 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:34.827973 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bfbcddb55-fnvw7: secret "image-registry-tls" not found Jul 09 04:00:34.828086 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:34.827989 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:00:34.828086 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:34.828024 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls podName:93ea36c7-0908-4ab9-8b33-fd167f9497ec nodeName:}" failed. No retries permitted until 2026-07-09 04:00:42.828008834 +0000 UTC m=+51.703926418 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls") pod "image-registry-bfbcddb55-fnvw7" (UID: "93ea36c7-0908-4ab9-8b33-fd167f9497ec") : secret "image-registry-tls" not found Jul 09 04:00:34.828086 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:34.828045 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls podName:d595e0d2-6955-4f99-9531-47b6c1ce95a3 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:42.828030275 +0000 UTC m=+51.703947844 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls") pod "dns-default-gcm68" (UID: "d595e0d2-6955-4f99-9531-47b6c1ce95a3") : secret "dns-default-metrics-tls" not found Jul 09 04:00:34.866440 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:34.866409 2576 generic.go:356] "Generic (PLEG): container finished" podID="a3544bd6-6306-4080-8a5f-d79d2dc9e69b" containerID="009e09e5382bba199eb8bd9d05248511f9e4244dc6c8b00be3ebc7d2b937d728" exitCode=0 Jul 09 04:00:34.866798 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:34.866464 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvsj4" event={"ID":"a3544bd6-6306-4080-8a5f-d79d2dc9e69b","Type":"ContainerDied","Data":"009e09e5382bba199eb8bd9d05248511f9e4244dc6c8b00be3ebc7d2b937d728"} Jul 09 04:00:35.871689 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:35.871654 2576 generic.go:356] "Generic (PLEG): container finished" podID="a3544bd6-6306-4080-8a5f-d79d2dc9e69b" containerID="3a7f648d035d1b15397b0fa2beb557ae3300fd783416a3b145acd2b080addc2f" exitCode=0 Jul 09 04:00:35.872087 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:35.871720 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvsj4" event={"ID":"a3544bd6-6306-4080-8a5f-d79d2dc9e69b","Type":"ContainerDied","Data":"3a7f648d035d1b15397b0fa2beb557ae3300fd783416a3b145acd2b080addc2f"} Jul 09 04:00:36.876762 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:36.876720 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rvsj4" event={"ID":"a3544bd6-6306-4080-8a5f-d79d2dc9e69b","Type":"ContainerStarted","Data":"e8f02004727d4269cf8902d0645d81f86dc983fac026aea9755fc674ca09b6f0"} Jul 09 04:00:36.903103 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:36.903053 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rvsj4" podStartSLOduration=8.608405939 podStartE2EDuration="44.903030604s" totalImagesPullingTime="36.294624665s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:59:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:00:36.901152467 +0000 UTC m=+45.777070069" watchObservedRunningTime="2026-07-09 04:00:36.903030604 +0000 UTC m=+45.778948195" Jul 09 04:00:42.884064 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:42.884019 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:42.884467 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:42.884080 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:42.884467 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:42.884168 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:00:42.884467 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:42.884170 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:00:42.884467 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:42.884192 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bfbcddb55-fnvw7: secret "image-registry-tls" not found Jul 09 04:00:42.884467 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:42.884221 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls podName:d595e0d2-6955-4f99-9531-47b6c1ce95a3 nodeName:}" failed. No retries permitted until 2026-07-09 04:00:58.88420621 +0000 UTC m=+67.760123776 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls") pod "dns-default-gcm68" (UID: "d595e0d2-6955-4f99-9531-47b6c1ce95a3") : secret "dns-default-metrics-tls" not found Jul 09 04:00:42.884467 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:42.884279 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls podName:93ea36c7-0908-4ab9-8b33-fd167f9497ec nodeName:}" failed. No retries permitted until 2026-07-09 04:00:58.884264898 +0000 UTC m=+67.760182463 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls") pod "image-registry-bfbcddb55-fnvw7" (UID: "93ea36c7-0908-4ab9-8b33-fd167f9497ec") : secret "image-registry-tls" not found Jul 09 04:00:52.830494 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:52.830463 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-974xm" Jul 09 04:00:56.380998 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:56.380956 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs\") pod \"network-metrics-daemon-7zmf5\" (UID: \"b13c1cbe-23d9-404e-b028-3efbf70d9fa0\") " pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:00:56.380998 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:56.381005 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret\") pod \"global-pull-secret-syncer-6mhqx\" (UID: \"68ab2626-7f66-43db-b616-996085b15324\") " pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:56.383736 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:56.383708 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 04:00:56.383910 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:56.383898 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 04:00:56.392111 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:56.392094 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 04:00:56.392193 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:56.392154 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs podName:b13c1cbe-23d9-404e-b028-3efbf70d9fa0 nodeName:}" failed. No retries permitted until 2026-07-09 04:02:00.392138286 +0000 UTC m=+129.268055852 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs") pod "network-metrics-daemon-7zmf5" (UID: "b13c1cbe-23d9-404e-b028-3efbf70d9fa0") : secret "metrics-daemon-secret" not found Jul 09 04:00:56.395232 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:56.395215 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/68ab2626-7f66-43db-b616-996085b15324-original-pull-secret\") pod \"global-pull-secret-syncer-6mhqx\" (UID: \"68ab2626-7f66-43db-b616-996085b15324\") " pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:56.481454 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:56.481417 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqr47\" (UniqueName: \"kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47\") pod \"network-check-target-f26vv\" (UID: \"e39b46e6-03eb-467c-876a-28c64869086f\") " pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:56.484491 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:56.484471 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 04:00:56.494076 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:56.494051 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 04:00:56.504565 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:56.504543 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqr47\" (UniqueName: \"kubernetes.io/projected/e39b46e6-03eb-467c-876a-28c64869086f-kube-api-access-vqr47\") pod \"network-check-target-f26vv\" (UID: \"e39b46e6-03eb-467c-876a-28c64869086f\") " pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:56.534274 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:56.534246 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-25smd\"" Jul 09 04:00:56.538495 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:56.538480 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6mhqx" Jul 09 04:00:56.541814 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:56.541799 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:00:56.666772 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:56.666740 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f26vv"] Jul 09 04:00:56.670048 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:00:56.670023 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode39b46e6_03eb_467c_876a_28c64869086f.slice/crio-a6156059b2c0bd2fe28d7fba18116c19a8c4239be900839d01b21930dc4ba341 WatchSource:0}: Error finding container a6156059b2c0bd2fe28d7fba18116c19a8c4239be900839d01b21930dc4ba341: Status 404 returned error can't find the container with id a6156059b2c0bd2fe28d7fba18116c19a8c4239be900839d01b21930dc4ba341 Jul 09 04:00:56.681737 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:56.681713 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6mhqx"] Jul 09 04:00:56.684402 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:00:56.684378 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68ab2626_7f66_43db_b616_996085b15324.slice/crio-59cc08bdff4acbfb1969f47e5759282335df7a3d9091f682aa4b72d9995f0494 WatchSource:0}: Error finding container 59cc08bdff4acbfb1969f47e5759282335df7a3d9091f682aa4b72d9995f0494: Status 404 returned error can't find the container with id 59cc08bdff4acbfb1969f47e5759282335df7a3d9091f682aa4b72d9995f0494 Jul 09 04:00:56.924997 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:56.923137 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f26vv" event={"ID":"e39b46e6-03eb-467c-876a-28c64869086f","Type":"ContainerStarted","Data":"a6156059b2c0bd2fe28d7fba18116c19a8c4239be900839d01b21930dc4ba341"} Jul 09 04:00:56.926165 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:56.926129 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6mhqx" event={"ID":"68ab2626-7f66-43db-b616-996085b15324","Type":"ContainerStarted","Data":"59cc08bdff4acbfb1969f47e5759282335df7a3d9091f682aa4b72d9995f0494"} Jul 09 04:00:58.900069 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:58.900033 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:00:58.900506 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:58.900212 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:00:58.900506 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:00:58.900213 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:00:58.900506 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:58.900287 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls podName:d595e0d2-6955-4f99-9531-47b6c1ce95a3 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:30.900265253 +0000 UTC m=+99.776182823 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls") pod "dns-default-gcm68" (UID: "d595e0d2-6955-4f99-9531-47b6c1ce95a3") : secret "dns-default-metrics-tls" not found Jul 09 04:00:58.900506 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:58.900291 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:00:58.900506 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:58.900308 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bfbcddb55-fnvw7: secret "image-registry-tls" not found Jul 09 04:00:58.900506 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:00:58.900359 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls podName:93ea36c7-0908-4ab9-8b33-fd167f9497ec nodeName:}" failed. No retries permitted until 2026-07-09 04:01:30.900340318 +0000 UTC m=+99.776257885 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls") pod "image-registry-bfbcddb55-fnvw7" (UID: "93ea36c7-0908-4ab9-8b33-fd167f9497ec") : secret "image-registry-tls" not found Jul 09 04:01:00.939287 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:00.939239 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f26vv" event={"ID":"e39b46e6-03eb-467c-876a-28c64869086f","Type":"ContainerStarted","Data":"640e9021f93f332cd52995ae16d296aa560cc905c7440285a658cce218d8025a"} Jul 09 04:01:00.939743 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:00.939416 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:01:00.964994 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:00.964934 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-f26vv" podStartSLOduration=65.614321134 podStartE2EDuration="1m8.9649124s" totalImagesPullingTime="3.350591266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:01:00.963613084 +0000 UTC m=+69.839530672" watchObservedRunningTime="2026-07-09 04:01:00.9649124 +0000 UTC m=+69.840829989" Jul 09 04:01:02.946049 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:02.946013 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6mhqx" event={"ID":"68ab2626-7f66-43db-b616-996085b15324","Type":"ContainerStarted","Data":"200a642d752214b159a67ceab3696b19bd59572958cb1d91297dcaa7fd07a33c"} Jul 09 04:01:02.961487 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:02.961439 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6mhqx" podStartSLOduration=65.614250533 podStartE2EDuration="1m10.961425176s" totalImagesPullingTime="5.347174643s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 03:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:01:02.961408321 +0000 UTC m=+71.837325900" watchObservedRunningTime="2026-07-09 04:01:02.961425176 +0000 UTC m=+71.837342763" Jul 09 04:01:18.614365 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.614243 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-dh5td"] Jul 09 04:01:18.618530 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.618513 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.621767 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.621740 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 09 04:01:18.634155 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.627240 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 04:01:18.634155 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.627562 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 09 04:01:18.634155 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.628806 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-29x9p\"" Jul 09 04:01:18.634155 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.631392 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 04:01:18.634155 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.633990 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-dh5td"] Jul 09 04:01:18.635593 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.634866 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 09 04:01:18.717770 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.717733 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk"] Jul 09 04:01:18.720918 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.720900 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-d8ppz"] Jul 09 04:01:18.721050 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.721034 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:18.724372 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.724350 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-d8ppz" Jul 09 04:01:18.725149 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.725131 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 09 04:01:18.726552 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.726529 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 04:01:18.726722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.726701 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kkppn\"" Jul 09 04:01:18.726825 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.726751 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 04:01:18.727413 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.727395 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 09 04:01:18.727503 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.727395 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 09 04:01:18.727998 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.727982 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-k66vg\"" Jul 09 04:01:18.728134 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.728119 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 09 04:01:18.740860 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.740814 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk"] Jul 09 04:01:18.741628 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.741608 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-d8ppz"] Jul 09 04:01:18.749305 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.749281 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-serving-cert\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.749419 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.749321 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.749419 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.749343 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-tmp\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.749419 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.749369 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-snapshots\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.749519 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.749431 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.749519 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.749486 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhr69\" (UniqueName: \"kubernetes.io/projected/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-kube-api-access-bhr69\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.850206 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.850161 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr85l\" (UniqueName: \"kubernetes.io/projected/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-kube-api-access-sr85l\") pod \"cluster-monitoring-operator-64786d9cbb-xrtbk\" (UID: \"f0082ed4-ae5f-4d63-9096-12bf3a7903fa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:18.850206 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.850210 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-serving-cert\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.850420 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.850228 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrh9n\" (UniqueName: \"kubernetes.io/projected/a6ba460a-0b9f-45d7-8bea-013f17b74d52-kube-api-access-jrh9n\") pod \"volume-data-source-validator-5d456fdbcd-d8ppz\" (UID: \"a6ba460a-0b9f-45d7-8bea-013f17b74d52\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-d8ppz" Jul 09 04:01:18.850420 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.850346 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.850420 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.850379 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xrtbk\" (UID: \"f0082ed4-ae5f-4d63-9096-12bf3a7903fa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:18.850420 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.850402 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-tmp\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.850582 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.850428 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-snapshots\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.850582 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.850455 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-xrtbk\" (UID: \"f0082ed4-ae5f-4d63-9096-12bf3a7903fa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:18.850582 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.850491 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.850712 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.850577 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhr69\" (UniqueName: \"kubernetes.io/projected/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-kube-api-access-bhr69\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.851061 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.851043 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.851061 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.851051 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-tmp\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.851164 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.851107 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-snapshots\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.851500 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.851483 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.852693 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.852669 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-serving-cert\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.858797 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.858773 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhr69\" (UniqueName: \"kubernetes.io/projected/45f40f4a-afe4-4007-8f0a-d2df2dfa9914-kube-api-access-bhr69\") pod \"insights-operator-5c847dcfc4-dh5td\" (UID: \"45f40f4a-afe4-4007-8f0a-d2df2dfa9914\") " pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.936330 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.936295 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" Jul 09 04:01:18.951688 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.951658 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr85l\" (UniqueName: \"kubernetes.io/projected/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-kube-api-access-sr85l\") pod \"cluster-monitoring-operator-64786d9cbb-xrtbk\" (UID: \"f0082ed4-ae5f-4d63-9096-12bf3a7903fa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:18.951923 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.951711 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrh9n\" (UniqueName: \"kubernetes.io/projected/a6ba460a-0b9f-45d7-8bea-013f17b74d52-kube-api-access-jrh9n\") pod \"volume-data-source-validator-5d456fdbcd-d8ppz\" (UID: \"a6ba460a-0b9f-45d7-8bea-013f17b74d52\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-d8ppz" Jul 09 04:01:18.951923 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.951757 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xrtbk\" (UID: \"f0082ed4-ae5f-4d63-9096-12bf3a7903fa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:18.951923 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.951786 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-xrtbk\" (UID: \"f0082ed4-ae5f-4d63-9096-12bf3a7903fa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:18.952107 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:18.952061 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 04:01:18.952168 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:18.952148 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls podName:f0082ed4-ae5f-4d63-9096-12bf3a7903fa nodeName:}" failed. No retries permitted until 2026-07-09 04:01:19.452127677 +0000 UTC m=+88.328045252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xrtbk" (UID: "f0082ed4-ae5f-4d63-9096-12bf3a7903fa") : secret "cluster-monitoring-operator-tls" not found Jul 09 04:01:18.952538 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.952517 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-xrtbk\" (UID: \"f0082ed4-ae5f-4d63-9096-12bf3a7903fa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:18.963553 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.963518 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrh9n\" (UniqueName: \"kubernetes.io/projected/a6ba460a-0b9f-45d7-8bea-013f17b74d52-kube-api-access-jrh9n\") pod \"volume-data-source-validator-5d456fdbcd-d8ppz\" (UID: \"a6ba460a-0b9f-45d7-8bea-013f17b74d52\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-d8ppz" Jul 09 04:01:18.964242 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:18.964218 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr85l\" (UniqueName: \"kubernetes.io/projected/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-kube-api-access-sr85l\") pod \"cluster-monitoring-operator-64786d9cbb-xrtbk\" (UID: \"f0082ed4-ae5f-4d63-9096-12bf3a7903fa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:19.037537 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:19.037508 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-d8ppz" Jul 09 04:01:19.068539 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:19.068494 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-dh5td"] Jul 09 04:01:19.072435 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:01:19.072401 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45f40f4a_afe4_4007_8f0a_d2df2dfa9914.slice/crio-03003ed0bf1d8467e4039e3a95268cbdfdefe53b32b4c68b9ad5446fc304f83b WatchSource:0}: Error finding container 03003ed0bf1d8467e4039e3a95268cbdfdefe53b32b4c68b9ad5446fc304f83b: Status 404 returned error can't find the container with id 03003ed0bf1d8467e4039e3a95268cbdfdefe53b32b4c68b9ad5446fc304f83b Jul 09 04:01:19.163005 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:19.162976 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-d8ppz"] Jul 09 04:01:19.165598 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:01:19.165566 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6ba460a_0b9f_45d7_8bea_013f17b74d52.slice/crio-198824d32a77b7ed647bc9b4a7cea6509c53f8db9006bce53aed51e6f647fcf7 WatchSource:0}: Error finding container 198824d32a77b7ed647bc9b4a7cea6509c53f8db9006bce53aed51e6f647fcf7: Status 404 returned error can't find the container with id 198824d32a77b7ed647bc9b4a7cea6509c53f8db9006bce53aed51e6f647fcf7 Jul 09 04:01:19.457271 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:19.457231 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xrtbk\" (UID: \"f0082ed4-ae5f-4d63-9096-12bf3a7903fa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:19.457444 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:19.457361 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 04:01:19.457444 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:19.457424 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls podName:f0082ed4-ae5f-4d63-9096-12bf3a7903fa nodeName:}" failed. No retries permitted until 2026-07-09 04:01:20.457410007 +0000 UTC m=+89.333327573 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xrtbk" (UID: "f0082ed4-ae5f-4d63-9096-12bf3a7903fa") : secret "cluster-monitoring-operator-tls" not found Jul 09 04:01:19.986858 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:19.986758 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" event={"ID":"45f40f4a-afe4-4007-8f0a-d2df2dfa9914","Type":"ContainerStarted","Data":"03003ed0bf1d8467e4039e3a95268cbdfdefe53b32b4c68b9ad5446fc304f83b"} Jul 09 04:01:19.988128 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:19.988098 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-d8ppz" event={"ID":"a6ba460a-0b9f-45d7-8bea-013f17b74d52","Type":"ContainerStarted","Data":"198824d32a77b7ed647bc9b4a7cea6509c53f8db9006bce53aed51e6f647fcf7"} Jul 09 04:01:20.465063 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:20.465023 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xrtbk\" (UID: \"f0082ed4-ae5f-4d63-9096-12bf3a7903fa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:20.465262 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:20.465164 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 04:01:20.465262 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:20.465251 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls podName:f0082ed4-ae5f-4d63-9096-12bf3a7903fa nodeName:}" failed. No retries permitted until 2026-07-09 04:01:22.465228658 +0000 UTC m=+91.341146228 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xrtbk" (UID: "f0082ed4-ae5f-4d63-9096-12bf3a7903fa") : secret "cluster-monitoring-operator-tls" not found Jul 09 04:01:20.991936 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:20.991902 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-d8ppz" event={"ID":"a6ba460a-0b9f-45d7-8bea-013f17b74d52","Type":"ContainerStarted","Data":"04197f2214c14945f06b610e2566af9bc293a22a50aa8309dbfbb9c48901bcb9"} Jul 09 04:01:21.900479 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:21.900416 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-d8ppz" podStartSLOduration=2.8176229040000003 podStartE2EDuration="3.900391428s" totalImagesPullingTime="1.082768524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:01:21.009191747 +0000 UTC m=+89.885109335" watchObservedRunningTime="2026-07-09 04:01:21.900391428 +0000 UTC m=+90.776309016" Jul 09 04:01:21.901194 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:21.901172 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bdhx7"] Jul 09 04:01:21.904360 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:21.904336 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:21.916213 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:21.916187 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 04:01:21.916335 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:21.916287 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9vrhl\"" Jul 09 04:01:21.940186 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:21.940156 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 04:01:21.967174 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:21.967139 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bdhx7"] Jul 09 04:01:21.977426 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:21.977390 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:21.977588 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:21.977458 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee942429-8ea2-4e5c-b56a-6b4510d15878-data-volume\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:21.977588 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:21.977489 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:21.977588 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:21.977521 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ee942429-8ea2-4e5c-b56a-6b4510d15878-crio-socket\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:21.977716 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:21.977632 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfdb6\" (UniqueName: \"kubernetes.io/projected/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-api-access-xfdb6\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:21.999045 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:21.999011 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" event={"ID":"45f40f4a-afe4-4007-8f0a-d2df2dfa9914","Type":"ContainerStarted","Data":"677a53bd61d7c381e9583b8006d599d5d7f16a45d794b686e3a6cbda9e3bd2e2"} Jul 09 04:01:22.019457 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:22.019403 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" podStartSLOduration=2.086987917 podStartE2EDuration="4.019385527s" totalImagesPullingTime="1.93239761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:01:22.018438137 +0000 UTC m=+90.894355727" watchObservedRunningTime="2026-07-09 04:01:22.019385527 +0000 UTC m=+90.895303112" Jul 09 04:01:22.078871 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:22.078823 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:22.079057 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:22.078906 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ee942429-8ea2-4e5c-b56a-6b4510d15878-crio-socket\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:22.079057 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:22.078940 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfdb6\" (UniqueName: \"kubernetes.io/projected/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-api-access-xfdb6\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:22.079057 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:22.078985 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:22.079057 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:22.079033 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:01:22.079057 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:22.079045 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ee942429-8ea2-4e5c-b56a-6b4510d15878-crio-socket\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:22.079057 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:22.079051 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:01:22.079492 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:22.079112 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls podName:ee942429-8ea2-4e5c-b56a-6b4510d15878 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:22.579089389 +0000 UTC m=+91.455006978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bdhx7" (UID: "ee942429-8ea2-4e5c-b56a-6b4510d15878") : secret "insights-runtime-extractor-tls" not found Jul 09 04:01:22.079492 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:22.079138 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm podName:ee942429-8ea2-4e5c-b56a-6b4510d15878 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:22.579123116 +0000 UTC m=+91.455040683 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bdhx7" (UID: "ee942429-8ea2-4e5c-b56a-6b4510d15878") : configmap "kube-rbac-proxy" not found Jul 09 04:01:22.079492 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:22.079281 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee942429-8ea2-4e5c-b56a-6b4510d15878-data-volume\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:22.079703 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:22.079680 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ee942429-8ea2-4e5c-b56a-6b4510d15878-data-volume\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:22.088204 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:22.088179 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfdb6\" (UniqueName: \"kubernetes.io/projected/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-api-access-xfdb6\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:22.482458 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:22.482417 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xrtbk\" (UID: \"f0082ed4-ae5f-4d63-9096-12bf3a7903fa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:22.482638 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:22.482548 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 04:01:22.482705 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:22.482685 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls podName:f0082ed4-ae5f-4d63-9096-12bf3a7903fa nodeName:}" failed. No retries permitted until 2026-07-09 04:01:26.482599491 +0000 UTC m=+95.358517057 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xrtbk" (UID: "f0082ed4-ae5f-4d63-9096-12bf3a7903fa") : secret "cluster-monitoring-operator-tls" not found Jul 09 04:01:22.583551 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:22.583495 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:22.583747 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:22.583587 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:22.583747 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:22.583652 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:01:22.583747 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:22.583670 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:01:22.583747 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:22.583721 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm podName:ee942429-8ea2-4e5c-b56a-6b4510d15878 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:23.583707771 +0000 UTC m=+92.459625350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bdhx7" (UID: "ee942429-8ea2-4e5c-b56a-6b4510d15878") : configmap "kube-rbac-proxy" not found Jul 09 04:01:22.583747 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:22.583734 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls podName:ee942429-8ea2-4e5c-b56a-6b4510d15878 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:23.58372774 +0000 UTC m=+92.459645305 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bdhx7" (UID: "ee942429-8ea2-4e5c-b56a-6b4510d15878") : secret "insights-runtime-extractor-tls" not found Jul 09 04:01:23.591182 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:23.591140 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:23.591589 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:23.591206 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:23.591589 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:23.591265 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:01:23.591589 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:23.591324 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:01:23.591589 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:23.591331 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm podName:ee942429-8ea2-4e5c-b56a-6b4510d15878 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:25.591317218 +0000 UTC m=+94.467234785 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bdhx7" (UID: "ee942429-8ea2-4e5c-b56a-6b4510d15878") : configmap "kube-rbac-proxy" not found Jul 09 04:01:23.591589 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:23.591362 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls podName:ee942429-8ea2-4e5c-b56a-6b4510d15878 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:25.591351157 +0000 UTC m=+94.467268723 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bdhx7" (UID: "ee942429-8ea2-4e5c-b56a-6b4510d15878") : secret "insights-runtime-extractor-tls" not found Jul 09 04:01:23.692581 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:23.692555 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-282rs_ef48e4fa-4394-40ac-8960-67646b4c0804/dns-node-resolver/0.log" Jul 09 04:01:24.692309 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:24.692280 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-55zxm_953f9350-02c8-4b0b-a2bb-28b305694ddb/node-ca/0.log" Jul 09 04:01:25.605396 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:25.605347 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:25.605590 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:25.605421 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:25.605590 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:25.605483 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:01:25.605590 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:25.605542 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:01:25.605590 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:25.605548 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm podName:ee942429-8ea2-4e5c-b56a-6b4510d15878 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:29.605532168 +0000 UTC m=+98.481449735 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bdhx7" (UID: "ee942429-8ea2-4e5c-b56a-6b4510d15878") : configmap "kube-rbac-proxy" not found Jul 09 04:01:25.605590 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:25.605584 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls podName:ee942429-8ea2-4e5c-b56a-6b4510d15878 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:29.605571346 +0000 UTC m=+98.481488911 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bdhx7" (UID: "ee942429-8ea2-4e5c-b56a-6b4510d15878") : secret "insights-runtime-extractor-tls" not found Jul 09 04:01:25.692291 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:25.692261 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p7z6b_53a12a15-f2c0-44ef-a267-6a540e093a33/serve-healthcheck-canary/0.log" Jul 09 04:01:26.311770 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.311736 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-x4x2k"] Jul 09 04:01:26.314866 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.314832 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-x4x2k" Jul 09 04:01:26.317606 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.317583 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 09 04:01:26.317606 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.317600 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 09 04:01:26.318738 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.318716 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 09 04:01:26.318828 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.318721 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-lxp9t\"" Jul 09 04:01:26.318828 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.318787 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 09 04:01:26.324068 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.324045 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-x4x2k"] Jul 09 04:01:26.411766 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.411731 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e8dc559d-dce3-495a-97d3-3a0b928debdb-signing-cabundle\") pod \"service-ca-758f599c5-x4x2k\" (UID: \"e8dc559d-dce3-495a-97d3-3a0b928debdb\") " pod="openshift-service-ca/service-ca-758f599c5-x4x2k" Jul 09 04:01:26.411958 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.411809 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e8dc559d-dce3-495a-97d3-3a0b928debdb-signing-key\") pod \"service-ca-758f599c5-x4x2k\" (UID: \"e8dc559d-dce3-495a-97d3-3a0b928debdb\") " pod="openshift-service-ca/service-ca-758f599c5-x4x2k" Jul 09 04:01:26.411958 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.411856 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t684h\" (UniqueName: \"kubernetes.io/projected/e8dc559d-dce3-495a-97d3-3a0b928debdb-kube-api-access-t684h\") pod \"service-ca-758f599c5-x4x2k\" (UID: \"e8dc559d-dce3-495a-97d3-3a0b928debdb\") " pod="openshift-service-ca/service-ca-758f599c5-x4x2k" Jul 09 04:01:26.512666 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.512628 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e8dc559d-dce3-495a-97d3-3a0b928debdb-signing-key\") pod \"service-ca-758f599c5-x4x2k\" (UID: \"e8dc559d-dce3-495a-97d3-3a0b928debdb\") " pod="openshift-service-ca/service-ca-758f599c5-x4x2k" Jul 09 04:01:26.512666 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.512672 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t684h\" (UniqueName: \"kubernetes.io/projected/e8dc559d-dce3-495a-97d3-3a0b928debdb-kube-api-access-t684h\") pod \"service-ca-758f599c5-x4x2k\" (UID: \"e8dc559d-dce3-495a-97d3-3a0b928debdb\") " pod="openshift-service-ca/service-ca-758f599c5-x4x2k" Jul 09 04:01:26.512964 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.512812 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xrtbk\" (UID: \"f0082ed4-ae5f-4d63-9096-12bf3a7903fa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:26.512964 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.512916 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e8dc559d-dce3-495a-97d3-3a0b928debdb-signing-cabundle\") pod \"service-ca-758f599c5-x4x2k\" (UID: \"e8dc559d-dce3-495a-97d3-3a0b928debdb\") " pod="openshift-service-ca/service-ca-758f599c5-x4x2k" Jul 09 04:01:26.513070 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:26.513010 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 04:01:26.513123 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:26.513091 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls podName:f0082ed4-ae5f-4d63-9096-12bf3a7903fa nodeName:}" failed. No retries permitted until 2026-07-09 04:01:34.513074164 +0000 UTC m=+103.388991734 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xrtbk" (UID: "f0082ed4-ae5f-4d63-9096-12bf3a7903fa") : secret "cluster-monitoring-operator-tls" not found Jul 09 04:01:26.513527 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.513504 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e8dc559d-dce3-495a-97d3-3a0b928debdb-signing-cabundle\") pod \"service-ca-758f599c5-x4x2k\" (UID: \"e8dc559d-dce3-495a-97d3-3a0b928debdb\") " pod="openshift-service-ca/service-ca-758f599c5-x4x2k" Jul 09 04:01:26.515006 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.514982 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e8dc559d-dce3-495a-97d3-3a0b928debdb-signing-key\") pod \"service-ca-758f599c5-x4x2k\" (UID: \"e8dc559d-dce3-495a-97d3-3a0b928debdb\") " pod="openshift-service-ca/service-ca-758f599c5-x4x2k" Jul 09 04:01:26.521987 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.521960 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t684h\" (UniqueName: \"kubernetes.io/projected/e8dc559d-dce3-495a-97d3-3a0b928debdb-kube-api-access-t684h\") pod \"service-ca-758f599c5-x4x2k\" (UID: \"e8dc559d-dce3-495a-97d3-3a0b928debdb\") " pod="openshift-service-ca/service-ca-758f599c5-x4x2k" Jul 09 04:01:26.624333 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.624251 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-x4x2k" Jul 09 04:01:26.755511 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:26.755482 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-x4x2k"] Jul 09 04:01:26.758529 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:01:26.758502 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8dc559d_dce3_495a_97d3_3a0b928debdb.slice/crio-3a275e2529ff310bd1761c6f3c87a3bcfe00894ba448384013b035fc3c855f09 WatchSource:0}: Error finding container 3a275e2529ff310bd1761c6f3c87a3bcfe00894ba448384013b035fc3c855f09: Status 404 returned error can't find the container with id 3a275e2529ff310bd1761c6f3c87a3bcfe00894ba448384013b035fc3c855f09 Jul 09 04:01:27.013528 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:27.013489 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-x4x2k" event={"ID":"e8dc559d-dce3-495a-97d3-3a0b928debdb","Type":"ContainerStarted","Data":"3a275e2529ff310bd1761c6f3c87a3bcfe00894ba448384013b035fc3c855f09"} Jul 09 04:01:29.021407 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:29.021312 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-x4x2k" event={"ID":"e8dc559d-dce3-495a-97d3-3a0b928debdb","Type":"ContainerStarted","Data":"4391da585b4f74804b216be762edab3e850877141f0721491d7c986b8285367c"} Jul 09 04:01:29.037785 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:29.037721 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-x4x2k" podStartSLOduration=1.095132721 podStartE2EDuration="3.03769747s" totalImagesPullingTime="1.942564749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:01:29.036693809 +0000 UTC m=+97.912611399" watchObservedRunningTime="2026-07-09 04:01:29.03769747 +0000 UTC m=+97.913615062" Jul 09 04:01:29.636920 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:29.636884 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:29.637102 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:29.636948 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:29.637102 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:29.637016 2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:01:29.637102 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:29.637048 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:01:29.637102 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:29.637098 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm podName:ee942429-8ea2-4e5c-b56a-6b4510d15878 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:37.637076683 +0000 UTC m=+106.512994280 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bdhx7" (UID: "ee942429-8ea2-4e5c-b56a-6b4510d15878") : configmap "kube-rbac-proxy" not found Jul 09 04:01:29.637242 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:29.637113 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls podName:ee942429-8ea2-4e5c-b56a-6b4510d15878 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:37.637106932 +0000 UTC m=+106.513024497 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bdhx7" (UID: "ee942429-8ea2-4e5c-b56a-6b4510d15878") : secret "insights-runtime-extractor-tls" not found Jul 09 04:01:30.947766 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:30.947731 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:01:30.948179 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:30.947810 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls\") pod \"image-registry-bfbcddb55-fnvw7\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:01:30.948179 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:30.947927 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:01:30.948179 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:30.947995 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls podName:d595e0d2-6955-4f99-9531-47b6c1ce95a3 nodeName:}" failed. No retries permitted until 2026-07-09 04:02:34.947981168 +0000 UTC m=+163.823898734 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls") pod "dns-default-gcm68" (UID: "d595e0d2-6955-4f99-9531-47b6c1ce95a3") : secret "dns-default-metrics-tls" not found Jul 09 04:01:30.948179 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:30.947940 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 04:01:30.948179 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:30.948031 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bfbcddb55-fnvw7: secret "image-registry-tls" not found Jul 09 04:01:30.948179 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:30.948072 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls podName:93ea36c7-0908-4ab9-8b33-fd167f9497ec nodeName:}" failed. No retries permitted until 2026-07-09 04:02:34.948060725 +0000 UTC m=+163.823978302 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls") pod "image-registry-bfbcddb55-fnvw7" (UID: "93ea36c7-0908-4ab9-8b33-fd167f9497ec") : secret "image-registry-tls" not found Jul 09 04:01:31.944982 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:31.944951 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-f26vv" Jul 09 04:01:34.583145 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:34.583100 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xrtbk\" (UID: \"f0082ed4-ae5f-4d63-9096-12bf3a7903fa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:34.583546 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:34.583223 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 04:01:34.583546 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:34.583276 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls podName:f0082ed4-ae5f-4d63-9096-12bf3a7903fa nodeName:}" failed. No retries permitted until 2026-07-09 04:01:50.583261728 +0000 UTC m=+119.459179294 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xrtbk" (UID: "f0082ed4-ae5f-4d63-9096-12bf3a7903fa") : secret "cluster-monitoring-operator-tls" not found Jul 09 04:01:37.708435 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:37.708400 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:37.709027 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:37.708465 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:37.709548 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:37.709508 2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:01:37.709804 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:01:37.709779 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls podName:ee942429-8ea2-4e5c-b56a-6b4510d15878 nodeName:}" failed. No retries permitted until 2026-07-09 04:01:53.709753933 +0000 UTC m=+122.585671504 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bdhx7" (UID: "ee942429-8ea2-4e5c-b56a-6b4510d15878") : secret "insights-runtime-extractor-tls" not found Jul 09 04:01:37.712767 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:37.710170 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ee942429-8ea2-4e5c-b56a-6b4510d15878-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:49.395833 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.395788 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56"] Jul 09 04:01:49.398826 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.398808 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.401497 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.401473 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-ca\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.401663 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.401504 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-hub\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.401663 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.401530 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.401663 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.401578 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.401663 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.401623 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.401663 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.401650 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h4hf\" (UniqueName: \"kubernetes.io/projected/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-kube-api-access-8h4hf\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.401947 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.401694 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 09 04:01:49.402298 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.402281 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 09 04:01:49.403265 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.403242 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 09 04:01:49.403368 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.403268 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 09 04:01:49.403368 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.403279 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 09 04:01:49.403581 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.403563 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 09 04:01:49.405603 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.405584 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 09 04:01:49.412462 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.412439 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56"] Jul 09 04:01:49.502860 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.502812 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-ca\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.503024 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.502866 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-hub\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.503024 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.502895 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.503024 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.502910 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.503387 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.503359 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.503503 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.503413 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h4hf\" (UniqueName: \"kubernetes.io/projected/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-kube-api-access-8h4hf\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.504004 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.503981 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.506695 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.506641 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.507248 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.507228 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-ca\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.508627 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.507921 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.509345 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.509324 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-hub\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.517081 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.517063 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h4hf\" (UniqueName: \"kubernetes.io/projected/53873ed1-0373-48ba-b5ea-2ccc5eb62a8e-kube-api-access-8h4hf\") pod \"cluster-proxy-proxy-agent-7d6888db86-v5x56\" (UID: \"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.717071 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.716988 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" Jul 09 04:01:49.844482 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:49.844448 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56"] Jul 09 04:01:49.847026 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:01:49.847000 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53873ed1_0373_48ba_b5ea_2ccc5eb62a8e.slice/crio-e865c222f496f0eae246dd189952647464e9d1af3f41fe51e1c4236917d1d2d1 WatchSource:0}: Error finding container e865c222f496f0eae246dd189952647464e9d1af3f41fe51e1c4236917d1d2d1: Status 404 returned error can't find the container with id e865c222f496f0eae246dd189952647464e9d1af3f41fe51e1c4236917d1d2d1 Jul 09 04:01:50.083895 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:50.083785 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" event={"ID":"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e","Type":"ContainerStarted","Data":"e865c222f496f0eae246dd189952647464e9d1af3f41fe51e1c4236917d1d2d1"} Jul 09 04:01:50.611121 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:50.611080 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xrtbk\" (UID: \"f0082ed4-ae5f-4d63-9096-12bf3a7903fa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:50.614058 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:50.614030 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0082ed4-ae5f-4d63-9096-12bf3a7903fa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xrtbk\" (UID: \"f0082ed4-ae5f-4d63-9096-12bf3a7903fa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:50.831802 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:50.831768 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" Jul 09 04:01:50.981518 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:50.981414 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk"] Jul 09 04:01:50.984887 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:01:50.984833 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0082ed4_ae5f_4d63_9096_12bf3a7903fa.slice/crio-296e4acbed334f5bc7604c27e73c17a7171c16c43d16f9278821745183255668 WatchSource:0}: Error finding container 296e4acbed334f5bc7604c27e73c17a7171c16c43d16f9278821745183255668: Status 404 returned error can't find the container with id 296e4acbed334f5bc7604c27e73c17a7171c16c43d16f9278821745183255668 Jul 09 04:01:51.087952 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:51.087912 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" event={"ID":"f0082ed4-ae5f-4d63-9096-12bf3a7903fa","Type":"ContainerStarted","Data":"296e4acbed334f5bc7604c27e73c17a7171c16c43d16f9278821745183255668"} Jul 09 04:01:53.740886 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:53.740741 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:53.743150 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:53.743120 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ee942429-8ea2-4e5c-b56a-6b4510d15878-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bdhx7\" (UID: \"ee942429-8ea2-4e5c-b56a-6b4510d15878\") " pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:54.018431 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:54.018349 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9vrhl\"" Jul 09 04:01:54.025986 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:54.025962 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bdhx7" Jul 09 04:01:54.103710 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:54.103641 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" event={"ID":"f0082ed4-ae5f-4d63-9096-12bf3a7903fa","Type":"ContainerStarted","Data":"b039f3b5b465e556203b244228c9f10c04866951914d3c2ac10f6d1b5b416714"} Jul 09 04:01:54.135429 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:54.135369 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" podStartSLOduration=33.721415905 podStartE2EDuration="36.135347106s" totalImagesPullingTime="2.413931201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:01:54.132523219 +0000 UTC m=+123.008440811" watchObservedRunningTime="2026-07-09 04:01:54.135347106 +0000 UTC m=+123.011264694" Jul 09 04:01:54.181746 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:54.181715 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bdhx7"] Jul 09 04:01:54.184943 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:01:54.184909 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee942429_8ea2_4e5c_b56a_6b4510d15878.slice/crio-8d4c55769152a992f31e0b43ff985bc79f6edaf07b8b72c3be88b82c36f745a6 WatchSource:0}: Error finding container 8d4c55769152a992f31e0b43ff985bc79f6edaf07b8b72c3be88b82c36f745a6: Status 404 returned error can't find the container with id 8d4c55769152a992f31e0b43ff985bc79f6edaf07b8b72c3be88b82c36f745a6 Jul 09 04:01:55.109971 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:55.109890 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bdhx7" event={"ID":"ee942429-8ea2-4e5c-b56a-6b4510d15878","Type":"ContainerStarted","Data":"8d4c55769152a992f31e0b43ff985bc79f6edaf07b8b72c3be88b82c36f745a6"} Jul 09 04:01:59.127554 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:59.127515 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bdhx7" event={"ID":"ee942429-8ea2-4e5c-b56a-6b4510d15878","Type":"ContainerStarted","Data":"9c400f420d9d57ed3bd1d821c83a0bfecb41e1bc400273f612d0f2c46ebe5aab"} Jul 09 04:01:59.129507 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:59.129481 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" event={"ID":"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e","Type":"ContainerStarted","Data":"a46e1cdae71d61a3d67c41dde8240c1427609790c7dedf38d703b01f8024e0e8"} Jul 09 04:01:59.129617 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:59.129515 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" event={"ID":"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e","Type":"ContainerStarted","Data":"72e4dd93cbc56c06122417dad3258ddd848f9e549068429483bec78a00db039b"} Jul 09 04:01:59.129617 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:59.129527 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" event={"ID":"53873ed1-0373-48ba-b5ea-2ccc5eb62a8e","Type":"ContainerStarted","Data":"84a549dfc518963ef4aa972288b411c1156b1bf6ca5caa28c95007bd1f4c94ad"} Jul 09 04:01:59.149921 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:01:59.149865 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6888db86-v5x56" podStartSLOduration=1.510476266 podStartE2EDuration="10.149828213s" totalImagesPullingTime="8.639351947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:01:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:01:59.14762762 +0000 UTC m=+128.023545226" watchObservedRunningTime="2026-07-09 04:01:59.149828213 +0000 UTC m=+128.025745808" Jul 09 04:02:00.135179 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:00.135124 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bdhx7" event={"ID":"ee942429-8ea2-4e5c-b56a-6b4510d15878","Type":"ContainerStarted","Data":"f53dd4da8bdaa3a5b92d7ee811b0e914df8b9063d521256ddbeecb2e930ca0f5"} Jul 09 04:02:00.401150 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:00.401115 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs\") pod \"network-metrics-daemon-7zmf5\" (UID: \"b13c1cbe-23d9-404e-b028-3efbf70d9fa0\") " pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:02:00.403733 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:00.403706 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b13c1cbe-23d9-404e-b028-3efbf70d9fa0-metrics-certs\") pod \"network-metrics-daemon-7zmf5\" (UID: \"b13c1cbe-23d9-404e-b028-3efbf70d9fa0\") " pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:02:00.447057 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:00.447024 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fn82r\"" Jul 09 04:02:00.455315 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:00.455287 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7zmf5" Jul 09 04:02:00.598194 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:00.598167 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7zmf5"] Jul 09 04:02:00.600623 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:02:00.600590 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb13c1cbe_23d9_404e_b028_3efbf70d9fa0.slice/crio-b2dfac1dbe597e1ac44cf93733f6cc0fadd71da490d8c2bbb4862df580557cd6 WatchSource:0}: Error finding container b2dfac1dbe597e1ac44cf93733f6cc0fadd71da490d8c2bbb4862df580557cd6: Status 404 returned error can't find the container with id b2dfac1dbe597e1ac44cf93733f6cc0fadd71da490d8c2bbb4862df580557cd6 Jul 09 04:02:01.139396 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:01.139361 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7zmf5" event={"ID":"b13c1cbe-23d9-404e-b028-3efbf70d9fa0","Type":"ContainerStarted","Data":"b2dfac1dbe597e1ac44cf93733f6cc0fadd71da490d8c2bbb4862df580557cd6"} Jul 09 04:02:02.144347 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:02.144314 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bdhx7" event={"ID":"ee942429-8ea2-4e5c-b56a-6b4510d15878","Type":"ContainerStarted","Data":"16adcc2f4c1f70bd4f85ad87af9d71b37c953a93571213b457be14544e5e5d37"} Jul 09 04:02:02.161434 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:02.161382 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bdhx7" podStartSLOduration=33.747145648 podStartE2EDuration="41.161366502s" totalImagesPullingTime="7.414220854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:01:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:02:02.160632299 +0000 UTC m=+131.036549888" watchObservedRunningTime="2026-07-09 04:02:02.161366502 +0000 UTC m=+131.037284090" Jul 09 04:02:03.149427 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:03.149390 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7zmf5" event={"ID":"b13c1cbe-23d9-404e-b028-3efbf70d9fa0","Type":"ContainerStarted","Data":"6bfa200bc913a998b1af3bcc39f73e0cc7dfcdc7881bb6af1920fa957ba3a52f"} Jul 09 04:02:03.149814 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:03.149450 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7zmf5" event={"ID":"b13c1cbe-23d9-404e-b028-3efbf70d9fa0","Type":"ContainerStarted","Data":"6cc65928eca5a11361336ac4f112b24f9102e0b058a317c48fbcb25009c4a65d"} Jul 09 04:02:03.166404 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:03.166355 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7zmf5" podStartSLOduration=129.1121181 podStartE2EDuration="2m11.166340942s" totalImagesPullingTime="2.054222842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 03:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:02:03.166215575 +0000 UTC m=+132.042133164" watchObservedRunningTime="2026-07-09 04:02:03.166340942 +0000 UTC m=+132.042258529" Jul 09 04:02:05.158159 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:05.158135 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:02:05.158510 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:05.158174 2576 generic.go:356] "Generic (PLEG): container finished" podID="f0082ed4-ae5f-4d63-9096-12bf3a7903fa" containerID="b039f3b5b465e556203b244228c9f10c04866951914d3c2ac10f6d1b5b416714" exitCode=2 Jul 09 04:02:05.158510 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:05.158230 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" event={"ID":"f0082ed4-ae5f-4d63-9096-12bf3a7903fa","Type":"ContainerDied","Data":"b039f3b5b465e556203b244228c9f10c04866951914d3c2ac10f6d1b5b416714"} Jul 09 04:02:05.158592 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:05.158528 2576 scope.go:117] "RemoveContainer" containerID="b039f3b5b465e556203b244228c9f10c04866951914d3c2ac10f6d1b5b416714" Jul 09 04:02:06.162995 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:06.162966 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:02:06.163368 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:06.163053 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xrtbk" event={"ID":"f0082ed4-ae5f-4d63-9096-12bf3a7903fa","Type":"ContainerStarted","Data":"c8abaf0b129735333583c6c57c9fd0d261f404d11e8e6341ee583858b3fb27a9"} Jul 09 04:02:08.929007 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:08.928963 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cnbzz"] Jul 09 04:02:08.932710 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:08.932684 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:08.936168 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:08.936127 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 04:02:08.936288 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:08.936171 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 04:02:08.936366 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:08.936336 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 04:02:08.936433 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:08.936409 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g2w2m\"" Jul 09 04:02:08.937486 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:08.937467 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 04:02:09.075011 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.074970 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.075011 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.075010 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d39f4d80-1a50-4766-998f-82afa4572f3a-sys\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.075280 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.075028 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-textfile\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.075280 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.075046 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-accelerators-collector-config\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.075280 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.075107 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-wtmp\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.075280 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.075141 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d39f4d80-1a50-4766-998f-82afa4572f3a-metrics-client-ca\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.075280 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.075178 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nkjm\" (UniqueName: \"kubernetes.io/projected/d39f4d80-1a50-4766-998f-82afa4572f3a-kube-api-access-8nkjm\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.075280 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.075202 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-tls\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.075280 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.075276 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d39f4d80-1a50-4766-998f-82afa4572f3a-root\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.176093 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.176047 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.176301 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.176164 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d39f4d80-1a50-4766-998f-82afa4572f3a-sys\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.176301 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.176183 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-textfile\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.176301 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.176200 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-accelerators-collector-config\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.176301 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.176227 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-wtmp\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.176301 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.176252 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d39f4d80-1a50-4766-998f-82afa4572f3a-metrics-client-ca\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.176301 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.176271 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d39f4d80-1a50-4766-998f-82afa4572f3a-sys\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.176301 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.176286 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nkjm\" (UniqueName: \"kubernetes.io/projected/d39f4d80-1a50-4766-998f-82afa4572f3a-kube-api-access-8nkjm\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.176656 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.176325 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-tls\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.176656 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.176381 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d39f4d80-1a50-4766-998f-82afa4572f3a-root\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.176656 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.176451 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d39f4d80-1a50-4766-998f-82afa4572f3a-root\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.176656 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.176519 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-textfile\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.176656 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:02:09.176545 2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 09 04:02:09.176656 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.176570 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-wtmp\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.176656 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:02:09.176639 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-tls podName:d39f4d80-1a50-4766-998f-82afa4572f3a nodeName:}" failed. No retries permitted until 2026-07-09 04:02:09.676612479 +0000 UTC m=+138.552530052 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-tls") pod "node-exporter-cnbzz" (UID: "d39f4d80-1a50-4766-998f-82afa4572f3a") : secret "node-exporter-tls" not found Jul 09 04:02:09.177080 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.177061 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d39f4d80-1a50-4766-998f-82afa4572f3a-metrics-client-ca\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.177131 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.177094 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-accelerators-collector-config\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.178475 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.178458 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.185146 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.185084 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nkjm\" (UniqueName: \"kubernetes.io/projected/d39f4d80-1a50-4766-998f-82afa4572f3a-kube-api-access-8nkjm\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.680981 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.680946 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-tls\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.683267 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.683245 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d39f4d80-1a50-4766-998f-82afa4572f3a-node-exporter-tls\") pod \"node-exporter-cnbzz\" (UID: \"d39f4d80-1a50-4766-998f-82afa4572f3a\") " pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.843092 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:09.843051 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cnbzz" Jul 09 04:02:09.856488 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:02:09.856447 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd39f4d80_1a50_4766_998f_82afa4572f3a.slice/crio-e942a5822fba78e60d18106a726bc724457d8fe73635722de2c8620c44cff8b9 WatchSource:0}: Error finding container e942a5822fba78e60d18106a726bc724457d8fe73635722de2c8620c44cff8b9: Status 404 returned error can't find the container with id e942a5822fba78e60d18106a726bc724457d8fe73635722de2c8620c44cff8b9 Jul 09 04:02:10.178645 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:10.178605 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cnbzz" event={"ID":"d39f4d80-1a50-4766-998f-82afa4572f3a","Type":"ContainerStarted","Data":"e942a5822fba78e60d18106a726bc724457d8fe73635722de2c8620c44cff8b9"} Jul 09 04:02:11.626619 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:11.626542 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-bfbcddb55-fnvw7"] Jul 09 04:02:11.627013 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:02:11.626759 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" podUID="93ea36c7-0908-4ab9-8b33-fd167f9497ec" Jul 09 04:02:12.187100 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.187066 2576 generic.go:356] "Generic (PLEG): container finished" podID="d39f4d80-1a50-4766-998f-82afa4572f3a" containerID="4c20620b6d1f54bc15f280fc0d155f731df25498162d2e3adad8a789b0cbbbcb" exitCode=0 Jul 09 04:02:12.187263 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.187158 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cnbzz" event={"ID":"d39f4d80-1a50-4766-998f-82afa4572f3a","Type":"ContainerDied","Data":"4c20620b6d1f54bc15f280fc0d155f731df25498162d2e3adad8a789b0cbbbcb"} Jul 09 04:02:12.187263 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.187175 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:02:12.193544 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.193524 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:02:12.305410 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.305385 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93ea36c7-0908-4ab9-8b33-fd167f9497ec-ca-trust-extracted\") pod \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " Jul 09 04:02:12.305504 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.305444 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93ea36c7-0908-4ab9-8b33-fd167f9497ec-trusted-ca\") pod \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " Jul 09 04:02:12.305557 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.305509 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-bound-sa-token\") pod \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " Jul 09 04:02:12.305557 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.305533 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-certificates\") pod \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " Jul 09 04:02:12.305697 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.305561 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93ea36c7-0908-4ab9-8b33-fd167f9497ec-installation-pull-secrets\") pod \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " Jul 09 04:02:12.305697 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.305607 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j65tc\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-kube-api-access-j65tc\") pod \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " Jul 09 04:02:12.305697 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.305631 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93ea36c7-0908-4ab9-8b33-fd167f9497ec-image-registry-private-configuration\") pod \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\" (UID: \"93ea36c7-0908-4ab9-8b33-fd167f9497ec\") " Jul 09 04:02:12.305697 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.305639 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93ea36c7-0908-4ab9-8b33-fd167f9497ec-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "93ea36c7-0908-4ab9-8b33-fd167f9497ec" (UID: "93ea36c7-0908-4ab9-8b33-fd167f9497ec"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:02:12.306013 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.305941 2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/93ea36c7-0908-4ab9-8b33-fd167f9497ec-ca-trust-extracted\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:02:12.306013 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.306001 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93ea36c7-0908-4ab9-8b33-fd167f9497ec-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "93ea36c7-0908-4ab9-8b33-fd167f9497ec" (UID: "93ea36c7-0908-4ab9-8b33-fd167f9497ec"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:02:12.306443 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.306407 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "93ea36c7-0908-4ab9-8b33-fd167f9497ec" (UID: "93ea36c7-0908-4ab9-8b33-fd167f9497ec"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:02:12.307927 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.307898 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-kube-api-access-j65tc" (OuterVolumeSpecName: "kube-api-access-j65tc") pod "93ea36c7-0908-4ab9-8b33-fd167f9497ec" (UID: "93ea36c7-0908-4ab9-8b33-fd167f9497ec"). InnerVolumeSpecName "kube-api-access-j65tc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:02:12.308324 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.308304 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93ea36c7-0908-4ab9-8b33-fd167f9497ec-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "93ea36c7-0908-4ab9-8b33-fd167f9497ec" (UID: "93ea36c7-0908-4ab9-8b33-fd167f9497ec"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:02:12.308423 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.308400 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "93ea36c7-0908-4ab9-8b33-fd167f9497ec" (UID: "93ea36c7-0908-4ab9-8b33-fd167f9497ec"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:02:12.308473 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.308439 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93ea36c7-0908-4ab9-8b33-fd167f9497ec-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "93ea36c7-0908-4ab9-8b33-fd167f9497ec" (UID: "93ea36c7-0908-4ab9-8b33-fd167f9497ec"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:02:12.406583 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.406545 2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-bound-sa-token\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:02:12.406583 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.406577 2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-certificates\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:02:12.406583 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.406586 2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/93ea36c7-0908-4ab9-8b33-fd167f9497ec-installation-pull-secrets\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:02:12.406583 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.406595 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j65tc\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-kube-api-access-j65tc\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:02:12.406887 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.406604 2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/93ea36c7-0908-4ab9-8b33-fd167f9497ec-image-registry-private-configuration\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:02:12.406887 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:12.406613 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/93ea36c7-0908-4ab9-8b33-fd167f9497ec-trusted-ca\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:02:13.193264 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.193238 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bfbcddb55-fnvw7" Jul 09 04:02:13.193735 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.193235 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cnbzz" event={"ID":"d39f4d80-1a50-4766-998f-82afa4572f3a","Type":"ContainerStarted","Data":"7a00d7da2904412a2bf975a0b7708636a89db585b15bfb191b433bb42d71ee81"} Jul 09 04:02:13.193735 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.193326 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cnbzz" event={"ID":"d39f4d80-1a50-4766-998f-82afa4572f3a","Type":"ContainerStarted","Data":"56e95fd63b2b1202ca567bfd5fa3dd1e5aadca421592ebda0256ec1d3418bf6f"} Jul 09 04:02:13.216761 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.216712 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cnbzz" podStartSLOduration=3.843002137 podStartE2EDuration="5.216697896s" totalImagesPullingTime="1.30670745s" totalInitContainerRuntime="66.988309ms" isStatefulPod=true podCreationTimestamp="2026-07-09 04:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:02:13.215174121 +0000 UTC m=+142.091091711" watchObservedRunningTime="2026-07-09 04:02:13.216697896 +0000 UTC m=+142.092615483" Jul 09 04:02:13.247074 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.247045 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-bfbcddb55-fnvw7"] Jul 09 04:02:13.250412 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.250384 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-bfbcddb55-fnvw7"] Jul 09 04:02:13.312105 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.312076 2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/93ea36c7-0908-4ab9-8b33-fd167f9497ec-registry-tls\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:02:13.622857 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.622760 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93ea36c7-0908-4ab9-8b33-fd167f9497ec" path="/var/lib/kubelet/pods/93ea36c7-0908-4ab9-8b33-fd167f9497ec/volumes" Jul 09 04:02:13.630988 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.630960 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-ml5xr"] Jul 09 04:02:13.635196 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.635175 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ml5xr" Jul 09 04:02:13.637793 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.637776 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-cqghv\"" Jul 09 04:02:13.637921 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.637773 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 09 04:02:13.642098 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.642074 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-ml5xr"] Jul 09 04:02:13.715425 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.715389 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/34cd756f-1a12-45c1-9787-05fc3cfd17ed-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-ml5xr\" (UID: \"34cd756f-1a12-45c1-9787-05fc3cfd17ed\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ml5xr" Jul 09 04:02:13.715425 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.715430 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmthn\" (UniqueName: \"kubernetes.io/projected/34cd756f-1a12-45c1-9787-05fc3cfd17ed-kube-api-access-bmthn\") pod \"monitoring-plugin-d5dcff7cf-ml5xr\" (UID: \"34cd756f-1a12-45c1-9787-05fc3cfd17ed\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ml5xr" Jul 09 04:02:13.816451 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.816413 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/34cd756f-1a12-45c1-9787-05fc3cfd17ed-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-ml5xr\" (UID: \"34cd756f-1a12-45c1-9787-05fc3cfd17ed\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ml5xr" Jul 09 04:02:13.816643 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.816460 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmthn\" (UniqueName: \"kubernetes.io/projected/34cd756f-1a12-45c1-9787-05fc3cfd17ed-kube-api-access-bmthn\") pod \"monitoring-plugin-d5dcff7cf-ml5xr\" (UID: \"34cd756f-1a12-45c1-9787-05fc3cfd17ed\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ml5xr" Jul 09 04:02:13.818941 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.818916 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/34cd756f-1a12-45c1-9787-05fc3cfd17ed-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-ml5xr\" (UID: \"34cd756f-1a12-45c1-9787-05fc3cfd17ed\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ml5xr" Jul 09 04:02:13.831182 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.831157 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmthn\" (UniqueName: \"kubernetes.io/projected/34cd756f-1a12-45c1-9787-05fc3cfd17ed-kube-api-access-bmthn\") pod \"monitoring-plugin-d5dcff7cf-ml5xr\" (UID: \"34cd756f-1a12-45c1-9787-05fc3cfd17ed\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ml5xr" Jul 09 04:02:13.945550 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:13.945522 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ml5xr" Jul 09 04:02:14.071598 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:14.071566 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-ml5xr"] Jul 09 04:02:14.075015 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:02:14.074984 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34cd756f_1a12_45c1_9787_05fc3cfd17ed.slice/crio-459e8b0926b8182aa38c41ef0d47f1d5befacaa266b37dbaee070e04cb3c8efa WatchSource:0}: Error finding container 459e8b0926b8182aa38c41ef0d47f1d5befacaa266b37dbaee070e04cb3c8efa: Status 404 returned error can't find the container with id 459e8b0926b8182aa38c41ef0d47f1d5befacaa266b37dbaee070e04cb3c8efa Jul 09 04:02:14.198043 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:14.197953 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ml5xr" event={"ID":"34cd756f-1a12-45c1-9787-05fc3cfd17ed","Type":"ContainerStarted","Data":"459e8b0926b8182aa38c41ef0d47f1d5befacaa266b37dbaee070e04cb3c8efa"} Jul 09 04:02:15.127593 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.127557 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:02:15.133031 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.133004 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.135975 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.135947 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7kc9r65ftpfcs\"" Jul 09 04:02:15.136122 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.136015 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 04:02:15.136214 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.136200 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 04:02:15.136267 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.136198 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 04:02:15.136491 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.136477 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 04:02:15.136732 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.136698 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 04:02:15.136732 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.136731 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 04:02:15.136922 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.136739 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 04:02:15.137261 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.137168 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 04:02:15.137261 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.137198 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 04:02:15.137261 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.137239 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 04:02:15.137450 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.137242 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 04:02:15.137512 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.137471 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bhn45\"" Jul 09 04:02:15.137512 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.137484 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 04:02:15.137705 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.137689 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 04:02:15.137789 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.137745 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 04:02:15.140875 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.140819 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 04:02:15.143709 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.142659 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:02:15.228218 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228185 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.228629 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228233 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.228629 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228260 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.228629 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228282 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.228629 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228351 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.228629 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228401 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.228629 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228434 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.228629 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228460 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-web-config\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.228629 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228481 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.228629 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228507 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.228629 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228534 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b49aa39d-c297-487e-8752-580d258daa34-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.228629 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228551 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.228629 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228582 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j7tq\" (UniqueName: \"kubernetes.io/projected/b49aa39d-c297-487e-8752-580d258daa34-kube-api-access-8j7tq\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.228629 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228598 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.229103 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228615 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.229103 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228643 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-config\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.229103 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228686 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b49aa39d-c297-487e-8752-580d258daa34-config-out\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.229103 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228712 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.229103 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228796 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.229103 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.228811 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.329965 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.329925 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.329965 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.329965 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.330230 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.329994 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.330230 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.330024 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.330230 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.330049 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.330230 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.330073 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.330230 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.330097 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.330230 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.330126 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.330737 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.330711 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.330881 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.330781 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.330881 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.330814 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-web-config\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.330881 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.330863 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.331045 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.330902 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.331045 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.330950 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b49aa39d-c297-487e-8752-580d258daa34-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.331045 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.330978 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.331045 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.331016 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j7tq\" (UniqueName: \"kubernetes.io/projected/b49aa39d-c297-487e-8752-580d258daa34-kube-api-access-8j7tq\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.331045 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.331042 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.331320 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.331041 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.331320 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.331071 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.331320 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.331124 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.332231 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.332201 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.332231 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.332206 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.332457 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.330901 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.333384 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.333351 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-config\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.333473 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.333410 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b49aa39d-c297-487e-8752-580d258daa34-config-out\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.333473 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.333439 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.334211 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.333694 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-web-config\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.334211 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.333715 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.334211 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.334121 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.334211 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.334158 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.334464 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.334236 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.334931 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.334884 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.335517 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.335468 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.335723 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.335702 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.335801 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.335749 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b49aa39d-c297-487e-8752-580d258daa34-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.336257 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.336236 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.336530 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.336507 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b49aa39d-c297-487e-8752-580d258daa34-config-out\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.336754 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.336737 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-config\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.336921 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.336904 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.339012 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.338993 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j7tq\" (UniqueName: \"kubernetes.io/projected/b49aa39d-c297-487e-8752-580d258daa34-kube-api-access-8j7tq\") pod \"prometheus-k8s-0\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.446120 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.446078 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:15.592159 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:15.592134 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:02:15.594712 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:02:15.594683 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb49aa39d_c297_487e_8752_580d258daa34.slice/crio-a052b4adddc2de8ded45c76b3fb55982ea125348a7925ccf428868aa271d1787 WatchSource:0}: Error finding container a052b4adddc2de8ded45c76b3fb55982ea125348a7925ccf428868aa271d1787: Status 404 returned error can't find the container with id a052b4adddc2de8ded45c76b3fb55982ea125348a7925ccf428868aa271d1787 Jul 09 04:02:16.207485 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:16.207441 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b49aa39d-c297-487e-8752-580d258daa34","Type":"ContainerStarted","Data":"a052b4adddc2de8ded45c76b3fb55982ea125348a7925ccf428868aa271d1787"} Jul 09 04:02:16.209057 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:16.209017 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ml5xr" event={"ID":"34cd756f-1a12-45c1-9787-05fc3cfd17ed","Type":"ContainerStarted","Data":"ed790d63b00db0840b1bc8305b82147ba24c1a6496791969a294a6f94323a41b"} Jul 09 04:02:16.209261 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:16.209240 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ml5xr" Jul 09 04:02:16.214945 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:16.214919 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ml5xr" Jul 09 04:02:16.231566 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:16.231513 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ml5xr" podStartSLOduration=2.06108922 podStartE2EDuration="3.231493928s" totalImagesPullingTime="1.170404708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:02:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:02:16.230263438 +0000 UTC m=+145.106181026" watchObservedRunningTime="2026-07-09 04:02:16.231493928 +0000 UTC m=+145.107411516" Jul 09 04:02:17.214050 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:17.214013 2576 generic.go:356] "Generic (PLEG): container finished" podID="b49aa39d-c297-487e-8752-580d258daa34" containerID="ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742" exitCode=0 Jul 09 04:02:17.214218 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:17.214100 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b49aa39d-c297-487e-8752-580d258daa34","Type":"ContainerDied","Data":"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742"} Jul 09 04:02:21.232130 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:21.232091 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b49aa39d-c297-487e-8752-580d258daa34","Type":"ContainerStarted","Data":"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21"} Jul 09 04:02:21.232494 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:21.232183 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b49aa39d-c297-487e-8752-580d258daa34","Type":"ContainerStarted","Data":"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814"} Jul 09 04:02:23.241283 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:23.241197 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b49aa39d-c297-487e-8752-580d258daa34","Type":"ContainerStarted","Data":"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63"} Jul 09 04:02:23.241283 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:23.241266 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b49aa39d-c297-487e-8752-580d258daa34","Type":"ContainerStarted","Data":"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871"} Jul 09 04:02:23.241674 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:23.241294 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b49aa39d-c297-487e-8752-580d258daa34","Type":"ContainerStarted","Data":"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd"} Jul 09 04:02:23.241674 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:23.241326 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b49aa39d-c297-487e-8752-580d258daa34","Type":"ContainerStarted","Data":"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e"} Jul 09 04:02:23.276264 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:23.276201 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.671360516 podStartE2EDuration="8.276179024s" totalImagesPullingTime="6.586079205s" totalInitContainerRuntime="18.739303ms" isStatefulPod=false podCreationTimestamp="2026-07-09 04:02:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:02:23.274997986 +0000 UTC m=+152.150915576" watchObservedRunningTime="2026-07-09 04:02:23.276179024 +0000 UTC m=+152.152096612" Jul 09 04:02:25.446618 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:25.446580 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:02:30.038549 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:02:30.038505 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-gcm68" podUID="d595e0d2-6955-4f99-9531-47b6c1ce95a3" Jul 09 04:02:30.268658 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:30.268628 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gcm68" Jul 09 04:02:35.008103 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:35.008068 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:02:35.010366 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:35.010342 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d595e0d2-6955-4f99-9531-47b6c1ce95a3-metrics-tls\") pod \"dns-default-gcm68\" (UID: \"d595e0d2-6955-4f99-9531-47b6c1ce95a3\") " pod="openshift-dns/dns-default-gcm68" Jul 09 04:02:35.072452 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:35.072424 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wgmf2\"" Jul 09 04:02:35.080226 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:35.080202 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gcm68" Jul 09 04:02:35.431550 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:35.431473 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gcm68"] Jul 09 04:02:35.434222 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:02:35.434195 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd595e0d2_6955_4f99_9531_47b6c1ce95a3.slice/crio-1d5df15996f081be2630344266d7df6a88e187fa9a6f15551b261aaac006be25 WatchSource:0}: Error finding container 1d5df15996f081be2630344266d7df6a88e187fa9a6f15551b261aaac006be25: Status 404 returned error can't find the container with id 1d5df15996f081be2630344266d7df6a88e187fa9a6f15551b261aaac006be25 Jul 09 04:02:36.293234 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:36.293187 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gcm68" event={"ID":"d595e0d2-6955-4f99-9531-47b6c1ce95a3","Type":"ContainerStarted","Data":"1d5df15996f081be2630344266d7df6a88e187fa9a6f15551b261aaac006be25"} Jul 09 04:02:38.302911 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:38.302868 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gcm68" event={"ID":"d595e0d2-6955-4f99-9531-47b6c1ce95a3","Type":"ContainerStarted","Data":"e3954bf00c44f3f4cb83c9c319b81553a0033c57fa4ccecac396ab30c5a6cf1d"} Jul 09 04:02:38.303300 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:38.302931 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gcm68" event={"ID":"d595e0d2-6955-4f99-9531-47b6c1ce95a3","Type":"ContainerStarted","Data":"569dbcdf68af6761a67c4b2abd5d0f68afaa986155124efc074bfcc385e572a9"} Jul 09 04:02:38.303300 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:38.302954 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gcm68" Jul 09 04:02:38.321227 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:38.321152 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gcm68" podStartSLOduration=130.312708556 podStartE2EDuration="2m12.321127912s" totalImagesPullingTime="2.008419356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:00:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:02:38.319228115 +0000 UTC m=+167.195145704" watchObservedRunningTime="2026-07-09 04:02:38.321127912 +0000 UTC m=+167.197045504" Jul 09 04:02:47.337707 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:47.337667 2576 generic.go:356] "Generic (PLEG): container finished" podID="45f40f4a-afe4-4007-8f0a-d2df2dfa9914" containerID="677a53bd61d7c381e9583b8006d599d5d7f16a45d794b686e3a6cbda9e3bd2e2" exitCode=0 Jul 09 04:02:47.338136 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:47.337747 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" event={"ID":"45f40f4a-afe4-4007-8f0a-d2df2dfa9914","Type":"ContainerDied","Data":"677a53bd61d7c381e9583b8006d599d5d7f16a45d794b686e3a6cbda9e3bd2e2"} Jul 09 04:02:47.338203 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:47.338184 2576 scope.go:117] "RemoveContainer" containerID="677a53bd61d7c381e9583b8006d599d5d7f16a45d794b686e3a6cbda9e3bd2e2" Jul 09 04:02:48.309748 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:48.309702 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gcm68" Jul 09 04:02:48.343097 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:02:48.343067 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-dh5td" event={"ID":"45f40f4a-afe4-4007-8f0a-d2df2dfa9914","Type":"ContainerStarted","Data":"de2e45f8aedf0dc99b94e16775bd3456b3e1783beffd69da00148f2866181eee"} Jul 09 04:03:15.446917 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:15.446881 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:15.466221 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:15.466196 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:16.463883 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:16.463830 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:33.453382 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.453347 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:03:33.453895 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.453639 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="prometheus" containerID="cri-o://afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814" gracePeriod=600 Jul 09 04:03:33.453895 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.453676 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="kube-rbac-proxy-web" containerID="cri-o://39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd" gracePeriod=600 Jul 09 04:03:33.453895 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.453639 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="kube-rbac-proxy" containerID="cri-o://cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871" gracePeriod=600 Jul 09 04:03:33.453895 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.453723 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="thanos-sidecar" containerID="cri-o://3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e" gracePeriod=600 Jul 09 04:03:33.453895 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.453657 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="kube-rbac-proxy-thanos" containerID="cri-o://a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63" gracePeriod=600 Jul 09 04:03:33.453895 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.453723 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="config-reloader" containerID="cri-o://7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21" gracePeriod=600 Jul 09 04:03:33.684936 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.684911 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:33.804369 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804273 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.804369 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804310 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-kube-rbac-proxy\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.804369 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804329 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-config\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.804647 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804371 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-trusted-ca-bundle\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.804647 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804413 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-tls\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.804647 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804450 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-0\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.804647 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804478 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-thanos-prometheus-http-client-file\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.804647 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804505 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8j7tq\" (UniqueName: \"kubernetes.io/projected/b49aa39d-c297-487e-8752-580d258daa34-kube-api-access-8j7tq\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.804647 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804535 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-serving-certs-ca-bundle\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.804647 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804563 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-kubelet-serving-ca-bundle\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.804647 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804592 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.804647 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804627 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-db\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.805122 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804659 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-2\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.805122 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804706 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-metrics-client-certs\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.805122 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804730 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-1\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.805122 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804755 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-metrics-client-ca\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.805122 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804793 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b49aa39d-c297-487e-8752-580d258daa34-config-out\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.805122 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804833 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-web-config\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.805122 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804898 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-grpc-tls\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.805122 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.804925 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b49aa39d-c297-487e-8752-580d258daa34-tls-assets\") pod \"b49aa39d-c297-487e-8752-580d258daa34\" (UID: \"b49aa39d-c297-487e-8752-580d258daa34\") " Jul 09 04:03:33.805122 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.805074 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:03:33.805122 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.805117 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:03:33.805577 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.805192 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-trusted-ca-bundle\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.805577 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.805212 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.805577 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.805506 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:03:33.805722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.805598 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:03:33.807095 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.805954 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:03:33.807095 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.805963 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:03:33.807095 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.807061 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:03:33.807602 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.807556 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:03:33.807918 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.807884 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:03:33.808423 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.808392 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-config" (OuterVolumeSpecName: "config") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:03:33.808748 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.808721 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:03:33.808830 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.808753 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:03:33.808830 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.808775 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:03:33.809002 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.808975 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:03:33.809213 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.809180 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:03:33.809308 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.809224 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b49aa39d-c297-487e-8752-580d258daa34-config-out" (OuterVolumeSpecName: "config-out") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:03:33.809745 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.809720 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:03:33.809824 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.809761 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b49aa39d-c297-487e-8752-580d258daa34-kube-api-access-8j7tq" (OuterVolumeSpecName: "kube-api-access-8j7tq") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "kube-api-access-8j7tq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:03:33.810194 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.810179 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b49aa39d-c297-487e-8752-580d258daa34-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:03:33.818661 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.818634 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-web-config" (OuterVolumeSpecName: "web-config") pod "b49aa39d-c297-487e-8752-580d258daa34" (UID: "b49aa39d-c297-487e-8752-580d258daa34"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:03:33.905884 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.905828 2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-web-config\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.905884 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.905878 2576 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-grpc-tls\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.905884 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.905888 2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b49aa39d-c297-487e-8752-580d258daa34-tls-assets\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.906111 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.905899 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.906111 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.905908 2576 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-kube-rbac-proxy\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.906111 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.905918 2576 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-config\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.906111 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.905928 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-tls\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.906111 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.905937 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.906111 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.905947 2576 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-thanos-prometheus-http-client-file\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.906111 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.905955 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8j7tq\" (UniqueName: \"kubernetes.io/projected/b49aa39d-c297-487e-8752-580d258daa34-kube-api-access-8j7tq\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.906111 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.905963 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.906111 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.905972 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.906111 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.905980 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-db\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.906111 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.905988 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.906111 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.905996 2576 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b49aa39d-c297-487e-8752-580d258daa34-secret-metrics-client-certs\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.906111 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.906004 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.906111 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.906012 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b49aa39d-c297-487e-8752-580d258daa34-configmap-metrics-client-ca\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:33.906111 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:33.906020 2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b49aa39d-c297-487e-8752-580d258daa34-config-out\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:03:34.516022 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.515988 2576 generic.go:356] "Generic (PLEG): container finished" podID="b49aa39d-c297-487e-8752-580d258daa34" containerID="a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63" exitCode=0 Jul 09 04:03:34.516022 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.516011 2576 generic.go:356] "Generic (PLEG): container finished" podID="b49aa39d-c297-487e-8752-580d258daa34" containerID="cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871" exitCode=0 Jul 09 04:03:34.516022 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.516017 2576 generic.go:356] "Generic (PLEG): container finished" podID="b49aa39d-c297-487e-8752-580d258daa34" containerID="39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd" exitCode=0 Jul 09 04:03:34.516022 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.516029 2576 generic.go:356] "Generic (PLEG): container finished" podID="b49aa39d-c297-487e-8752-580d258daa34" containerID="3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e" exitCode=0 Jul 09 04:03:34.516632 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.516038 2576 generic.go:356] "Generic (PLEG): container finished" podID="b49aa39d-c297-487e-8752-580d258daa34" containerID="7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21" exitCode=0 Jul 09 04:03:34.516632 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.516043 2576 generic.go:356] "Generic (PLEG): container finished" podID="b49aa39d-c297-487e-8752-580d258daa34" containerID="afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814" exitCode=0 Jul 09 04:03:34.516632 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.516069 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.516632 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.516073 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b49aa39d-c297-487e-8752-580d258daa34","Type":"ContainerDied","Data":"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63"} Jul 09 04:03:34.516632 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.516172 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b49aa39d-c297-487e-8752-580d258daa34","Type":"ContainerDied","Data":"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871"} Jul 09 04:03:34.516632 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.516214 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b49aa39d-c297-487e-8752-580d258daa34","Type":"ContainerDied","Data":"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd"} Jul 09 04:03:34.516632 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.516233 2576 scope.go:117] "RemoveContainer" containerID="a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63" Jul 09 04:03:34.516632 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.516257 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b49aa39d-c297-487e-8752-580d258daa34","Type":"ContainerDied","Data":"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e"} Jul 09 04:03:34.516632 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.516301 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b49aa39d-c297-487e-8752-580d258daa34","Type":"ContainerDied","Data":"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21"} Jul 09 04:03:34.516632 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.516341 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b49aa39d-c297-487e-8752-580d258daa34","Type":"ContainerDied","Data":"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814"} Jul 09 04:03:34.516632 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.516379 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b49aa39d-c297-487e-8752-580d258daa34","Type":"ContainerDied","Data":"a052b4adddc2de8ded45c76b3fb55982ea125348a7925ccf428868aa271d1787"} Jul 09 04:03:34.524187 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.524168 2576 scope.go:117] "RemoveContainer" containerID="cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871" Jul 09 04:03:34.530965 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.530948 2576 scope.go:117] "RemoveContainer" containerID="39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd" Jul 09 04:03:34.537689 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.537673 2576 scope.go:117] "RemoveContainer" containerID="3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e" Jul 09 04:03:34.539890 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.539864 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:03:34.544604 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.544584 2576 scope.go:117] "RemoveContainer" containerID="7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21" Jul 09 04:03:34.547167 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.547141 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:03:34.551545 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.551527 2576 scope.go:117] "RemoveContainer" containerID="afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814" Jul 09 04:03:34.560320 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.560296 2576 scope.go:117] "RemoveContainer" containerID="ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742" Jul 09 04:03:34.566976 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.566956 2576 scope.go:117] "RemoveContainer" containerID="a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63" Jul 09 04:03:34.567228 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:03:34.567207 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63\": container with ID starting with a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63 not found: ID does not exist" containerID="a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63" Jul 09 04:03:34.567275 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.567238 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63"} err="failed to get container status \"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63\": rpc error: code = NotFound desc = could not find container \"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63\": container with ID starting with a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63 not found: ID does not exist" Jul 09 04:03:34.567275 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.567273 2576 scope.go:117] "RemoveContainer" containerID="cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871" Jul 09 04:03:34.567513 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:03:34.567497 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871\": container with ID starting with cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871 not found: ID does not exist" containerID="cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871" Jul 09 04:03:34.567554 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.567516 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871"} err="failed to get container status \"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871\": rpc error: code = NotFound desc = could not find container \"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871\": container with ID starting with cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871 not found: ID does not exist" Jul 09 04:03:34.567554 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.567529 2576 scope.go:117] "RemoveContainer" containerID="39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd" Jul 09 04:03:34.567725 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:03:34.567708 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd\": container with ID starting with 39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd not found: ID does not exist" containerID="39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd" Jul 09 04:03:34.567763 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.567731 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd"} err="failed to get container status \"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd\": rpc error: code = NotFound desc = could not find container \"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd\": container with ID starting with 39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd not found: ID does not exist" Jul 09 04:03:34.567763 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.567746 2576 scope.go:117] "RemoveContainer" containerID="3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e" Jul 09 04:03:34.568221 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:03:34.568202 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e\": container with ID starting with 3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e not found: ID does not exist" containerID="3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e" Jul 09 04:03:34.568269 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.568226 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e"} err="failed to get container status \"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e\": rpc error: code = NotFound desc = could not find container \"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e\": container with ID starting with 3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e not found: ID does not exist" Jul 09 04:03:34.568269 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.568240 2576 scope.go:117] "RemoveContainer" containerID="7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21" Jul 09 04:03:34.568439 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:03:34.568423 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21\": container with ID starting with 7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21 not found: ID does not exist" containerID="7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21" Jul 09 04:03:34.568479 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.568444 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21"} err="failed to get container status \"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21\": rpc error: code = NotFound desc = could not find container \"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21\": container with ID starting with 7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21 not found: ID does not exist" Jul 09 04:03:34.568479 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.568458 2576 scope.go:117] "RemoveContainer" containerID="afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814" Jul 09 04:03:34.568687 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:03:34.568673 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814\": container with ID starting with afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814 not found: ID does not exist" containerID="afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814" Jul 09 04:03:34.568727 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.568689 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814"} err="failed to get container status \"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814\": rpc error: code = NotFound desc = could not find container \"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814\": container with ID starting with afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814 not found: ID does not exist" Jul 09 04:03:34.568727 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.568701 2576 scope.go:117] "RemoveContainer" containerID="ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742" Jul 09 04:03:34.568949 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:03:34.568930 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742\": container with ID starting with ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742 not found: ID does not exist" containerID="ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742" Jul 09 04:03:34.569003 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.568952 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742"} err="failed to get container status \"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742\": rpc error: code = NotFound desc = could not find container \"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742\": container with ID starting with ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742 not found: ID does not exist" Jul 09 04:03:34.569003 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.568968 2576 scope.go:117] "RemoveContainer" containerID="a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63" Jul 09 04:03:34.569192 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.569176 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63"} err="failed to get container status \"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63\": rpc error: code = NotFound desc = could not find container \"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63\": container with ID starting with a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63 not found: ID does not exist" Jul 09 04:03:34.569192 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.569192 2576 scope.go:117] "RemoveContainer" containerID="cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871" Jul 09 04:03:34.569394 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.569379 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871"} err="failed to get container status \"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871\": rpc error: code = NotFound desc = could not find container \"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871\": container with ID starting with cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871 not found: ID does not exist" Jul 09 04:03:34.569436 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.569394 2576 scope.go:117] "RemoveContainer" containerID="39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd" Jul 09 04:03:34.569571 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.569555 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd"} err="failed to get container status \"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd\": rpc error: code = NotFound desc = could not find container \"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd\": container with ID starting with 39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd not found: ID does not exist" Jul 09 04:03:34.569611 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.569573 2576 scope.go:117] "RemoveContainer" containerID="3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e" Jul 09 04:03:34.569734 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.569719 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e"} err="failed to get container status \"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e\": rpc error: code = NotFound desc = could not find container \"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e\": container with ID starting with 3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e not found: ID does not exist" Jul 09 04:03:34.569771 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.569734 2576 scope.go:117] "RemoveContainer" containerID="7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21" Jul 09 04:03:34.569925 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.569907 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21"} err="failed to get container status \"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21\": rpc error: code = NotFound desc = could not find container \"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21\": container with ID starting with 7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21 not found: ID does not exist" Jul 09 04:03:34.569970 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.569927 2576 scope.go:117] "RemoveContainer" containerID="afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814" Jul 09 04:03:34.570104 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.570084 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814"} err="failed to get container status \"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814\": rpc error: code = NotFound desc = could not find container \"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814\": container with ID starting with afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814 not found: ID does not exist" Jul 09 04:03:34.570175 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.570106 2576 scope.go:117] "RemoveContainer" containerID="ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742" Jul 09 04:03:34.570367 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.570351 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742"} err="failed to get container status \"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742\": rpc error: code = NotFound desc = could not find container \"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742\": container with ID starting with ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742 not found: ID does not exist" Jul 09 04:03:34.570411 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.570368 2576 scope.go:117] "RemoveContainer" containerID="a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63" Jul 09 04:03:34.570605 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.570586 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63"} err="failed to get container status \"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63\": rpc error: code = NotFound desc = could not find container \"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63\": container with ID starting with a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63 not found: ID does not exist" Jul 09 04:03:34.570667 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.570606 2576 scope.go:117] "RemoveContainer" containerID="cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871" Jul 09 04:03:34.570824 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.570806 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871"} err="failed to get container status \"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871\": rpc error: code = NotFound desc = could not find container \"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871\": container with ID starting with cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871 not found: ID does not exist" Jul 09 04:03:34.570884 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.570825 2576 scope.go:117] "RemoveContainer" containerID="39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd" Jul 09 04:03:34.571034 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.571018 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd"} err="failed to get container status \"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd\": rpc error: code = NotFound desc = could not find container \"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd\": container with ID starting with 39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd not found: ID does not exist" Jul 09 04:03:34.571074 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.571034 2576 scope.go:117] "RemoveContainer" containerID="3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e" Jul 09 04:03:34.571248 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.571232 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e"} err="failed to get container status \"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e\": rpc error: code = NotFound desc = could not find container \"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e\": container with ID starting with 3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e not found: ID does not exist" Jul 09 04:03:34.576722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.571248 2576 scope.go:117] "RemoveContainer" containerID="7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21" Jul 09 04:03:34.576722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.574244 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21"} err="failed to get container status \"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21\": rpc error: code = NotFound desc = could not find container \"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21\": container with ID starting with 7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21 not found: ID does not exist" Jul 09 04:03:34.576722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.574271 2576 scope.go:117] "RemoveContainer" containerID="afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814" Jul 09 04:03:34.576722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.574545 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814"} err="failed to get container status \"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814\": rpc error: code = NotFound desc = could not find container \"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814\": container with ID starting with afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814 not found: ID does not exist" Jul 09 04:03:34.576722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.574567 2576 scope.go:117] "RemoveContainer" containerID="ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742" Jul 09 04:03:34.576722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.574819 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742"} err="failed to get container status \"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742\": rpc error: code = NotFound desc = could not find container \"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742\": container with ID starting with ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742 not found: ID does not exist" Jul 09 04:03:34.576722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.574861 2576 scope.go:117] "RemoveContainer" containerID="a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63" Jul 09 04:03:34.576722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.575134 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63"} err="failed to get container status \"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63\": rpc error: code = NotFound desc = could not find container \"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63\": container with ID starting with a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63 not found: ID does not exist" Jul 09 04:03:34.576722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.575176 2576 scope.go:117] "RemoveContainer" containerID="cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871" Jul 09 04:03:34.576722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.575494 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871"} err="failed to get container status \"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871\": rpc error: code = NotFound desc = could not find container \"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871\": container with ID starting with cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871 not found: ID does not exist" Jul 09 04:03:34.576722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.575516 2576 scope.go:117] "RemoveContainer" containerID="39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd" Jul 09 04:03:34.576722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.575752 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd"} err="failed to get container status \"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd\": rpc error: code = NotFound desc = could not find container \"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd\": container with ID starting with 39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd not found: ID does not exist" Jul 09 04:03:34.576722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.575780 2576 scope.go:117] "RemoveContainer" containerID="3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e" Jul 09 04:03:34.576722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.575961 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:03:34.576722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576046 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e"} err="failed to get container status \"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e\": rpc error: code = NotFound desc = could not find container \"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e\": container with ID starting with 3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e not found: ID does not exist" Jul 09 04:03:34.576722 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576073 2576 scope.go:117] "RemoveContainer" containerID="7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576318 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21"} err="failed to get container status \"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21\": rpc error: code = NotFound desc = could not find container \"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21\": container with ID starting with 7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21 not found: ID does not exist" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576346 2576 scope.go:117] "RemoveContainer" containerID="afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576455 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="init-config-reloader" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576470 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="init-config-reloader" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576494 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="prometheus" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576503 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="prometheus" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576517 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="kube-rbac-proxy" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576526 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="kube-rbac-proxy" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576537 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="thanos-sidecar" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576546 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="thanos-sidecar" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576560 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="kube-rbac-proxy-web" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576568 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="kube-rbac-proxy-web" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576588 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814"} err="failed to get container status \"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814\": rpc error: code = NotFound desc = could not find container \"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814\": container with ID starting with afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814 not found: ID does not exist" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576610 2576 scope.go:117] "RemoveContainer" containerID="ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576599 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="config-reloader" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576702 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="config-reloader" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576783 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="kube-rbac-proxy-thanos" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.576800 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="kube-rbac-proxy-thanos" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.577258 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="config-reloader" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.577270 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="thanos-sidecar" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.577278 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="kube-rbac-proxy-thanos" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.577285 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="prometheus" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.577290 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="kube-rbac-proxy-web" Jul 09 04:03:34.577338 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.577301 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b49aa39d-c297-487e-8752-580d258daa34" containerName="kube-rbac-proxy" Jul 09 04:03:34.578212 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.577301 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742"} err="failed to get container status \"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742\": rpc error: code = NotFound desc = could not find container \"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742\": container with ID starting with ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742 not found: ID does not exist" Jul 09 04:03:34.578212 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.577326 2576 scope.go:117] "RemoveContainer" containerID="a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63" Jul 09 04:03:34.578212 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.577616 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63"} err="failed to get container status \"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63\": rpc error: code = NotFound desc = could not find container \"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63\": container with ID starting with a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63 not found: ID does not exist" Jul 09 04:03:34.578212 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.577638 2576 scope.go:117] "RemoveContainer" containerID="cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871" Jul 09 04:03:34.578212 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.577895 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871"} err="failed to get container status \"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871\": rpc error: code = NotFound desc = could not find container \"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871\": container with ID starting with cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871 not found: ID does not exist" Jul 09 04:03:34.578212 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.577911 2576 scope.go:117] "RemoveContainer" containerID="39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd" Jul 09 04:03:34.578212 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.578128 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd"} err="failed to get container status \"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd\": rpc error: code = NotFound desc = could not find container \"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd\": container with ID starting with 39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd not found: ID does not exist" Jul 09 04:03:34.578212 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.578143 2576 scope.go:117] "RemoveContainer" containerID="3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e" Jul 09 04:03:34.578507 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.578339 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e"} err="failed to get container status \"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e\": rpc error: code = NotFound desc = could not find container \"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e\": container with ID starting with 3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e not found: ID does not exist" Jul 09 04:03:34.578507 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.578369 2576 scope.go:117] "RemoveContainer" containerID="7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21" Jul 09 04:03:34.578638 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.578618 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21"} err="failed to get container status \"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21\": rpc error: code = NotFound desc = could not find container \"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21\": container with ID starting with 7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21 not found: ID does not exist" Jul 09 04:03:34.578710 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.578641 2576 scope.go:117] "RemoveContainer" containerID="afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814" Jul 09 04:03:34.578958 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.578934 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814"} err="failed to get container status \"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814\": rpc error: code = NotFound desc = could not find container \"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814\": container with ID starting with afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814 not found: ID does not exist" Jul 09 04:03:34.579094 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.578964 2576 scope.go:117] "RemoveContainer" containerID="ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742" Jul 09 04:03:34.579248 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.579223 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742"} err="failed to get container status \"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742\": rpc error: code = NotFound desc = could not find container \"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742\": container with ID starting with ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742 not found: ID does not exist" Jul 09 04:03:34.579248 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.579241 2576 scope.go:117] "RemoveContainer" containerID="a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63" Jul 09 04:03:34.579448 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.579430 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63"} err="failed to get container status \"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63\": rpc error: code = NotFound desc = could not find container \"a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63\": container with ID starting with a17ec97c74fe9f625c9c4ac4ab5562a13bbb1d0160ef683cbd66359bf2e6df63 not found: ID does not exist" Jul 09 04:03:34.579491 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.579450 2576 scope.go:117] "RemoveContainer" containerID="cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871" Jul 09 04:03:34.579664 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.579647 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871"} err="failed to get container status \"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871\": rpc error: code = NotFound desc = could not find container \"cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871\": container with ID starting with cb81abc84a59e7bd5cacc329a392cfefb76fd5d347262d4e44ed800501ece871 not found: ID does not exist" Jul 09 04:03:34.579707 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.579665 2576 scope.go:117] "RemoveContainer" containerID="39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd" Jul 09 04:03:34.579917 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.579898 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd"} err="failed to get container status \"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd\": rpc error: code = NotFound desc = could not find container \"39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd\": container with ID starting with 39cb1680631f41fb4a53674a20dccc4b2f155915d3067fc0303bec361d7fb3bd not found: ID does not exist" Jul 09 04:03:34.579989 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.579918 2576 scope.go:117] "RemoveContainer" containerID="3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e" Jul 09 04:03:34.580125 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.580109 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e"} err="failed to get container status \"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e\": rpc error: code = NotFound desc = could not find container \"3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e\": container with ID starting with 3aebf55829e51d29a83cfacf782b29c841d1e8bb4a9a833395943279e8ecc05e not found: ID does not exist" Jul 09 04:03:34.580188 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.580126 2576 scope.go:117] "RemoveContainer" containerID="7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21" Jul 09 04:03:34.580346 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.580330 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21"} err="failed to get container status \"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21\": rpc error: code = NotFound desc = could not find container \"7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21\": container with ID starting with 7a5d9e72e38ce4a5989f006a28e826ac696b79d86e37dae1661ef0c5518e2e21 not found: ID does not exist" Jul 09 04:03:34.580392 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.580347 2576 scope.go:117] "RemoveContainer" containerID="afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814" Jul 09 04:03:34.580528 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.580511 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814"} err="failed to get container status \"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814\": rpc error: code = NotFound desc = could not find container \"afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814\": container with ID starting with afa4c92a2a3e2e29228d999f735c4d0ac087cf3cb1442570e9db8ef9331bd814 not found: ID does not exist" Jul 09 04:03:34.580593 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.580529 2576 scope.go:117] "RemoveContainer" containerID="ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742" Jul 09 04:03:34.580726 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.580708 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742"} err="failed to get container status \"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742\": rpc error: code = NotFound desc = could not find container \"ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742\": container with ID starting with ecada8e5cc7d29c2093aa68b5d9d28dd51dae639ab78de1119d0912879792742 not found: ID does not exist" Jul 09 04:03:34.582717 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.582703 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.587711 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.587691 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 09 04:03:34.587811 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.587691 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7kc9r65ftpfcs\"" Jul 09 04:03:34.587975 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.587959 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 09 04:03:34.589175 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.589158 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 09 04:03:34.589550 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.589522 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 09 04:03:34.589652 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.589585 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 09 04:03:34.589706 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.589679 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 09 04:03:34.589706 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.589696 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 09 04:03:34.589833 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.589816 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 09 04:03:34.589905 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.589862 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bhn45\"" Jul 09 04:03:34.590145 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.590067 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 09 04:03:34.590317 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.590300 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 09 04:03:34.590386 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.590331 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 09 04:03:34.591100 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.591085 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 09 04:03:34.592481 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.592467 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 09 04:03:34.592938 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.592918 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 09 04:03:34.598291 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.598269 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 09 04:03:34.598593 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.598574 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:03:34.711162 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711122 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711162 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711159 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711415 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711177 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711415 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711193 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711415 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711209 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711415 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711236 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711415 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711296 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711415 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711345 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711415 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711370 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711677 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711430 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711677 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711459 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711677 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711488 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711677 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711505 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711677 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711520 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711677 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711656 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711965 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711681 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-config\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711965 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711707 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711965 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711749 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711965 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711774 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.711965 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.711906 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhw4q\" (UniqueName: \"kubernetes.io/projected/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-kube-api-access-zhw4q\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813073 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.812990 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813073 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813029 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813073 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813050 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhw4q\" (UniqueName: \"kubernetes.io/projected/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-kube-api-access-zhw4q\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813073 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813074 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813367 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813089 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813367 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813109 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813367 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813133 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813513 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813442 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813513 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813484 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813617 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813519 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813617 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813552 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813617 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813582 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813769 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813620 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813769 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813651 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813769 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813705 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813769 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813736 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.813769 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813764 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.814064 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813799 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.814064 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813822 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-config\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.814064 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813888 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.814064 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.813949 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.814257 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.814214 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.814482 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.814455 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.815006 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.814979 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.815811 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.815617 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.816749 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.816139 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.816749 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.816202 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.816749 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.816261 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.816749 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.816500 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.816749 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.816636 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.817558 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.817531 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.817700 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.817678 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.818115 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.818086 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.818249 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.818206 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.818603 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.818577 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.818672 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.818612 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.818828 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.818812 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.819052 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.819033 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-config\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.819261 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.819247 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.822488 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.822466 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhw4q\" (UniqueName: \"kubernetes.io/projected/592a60a4-9e18-4329-9f82-ddd3fd72c8a1-kube-api-access-zhw4q\") pod \"prometheus-k8s-0\" (UID: \"592a60a4-9e18-4329-9f82-ddd3fd72c8a1\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:34.892517 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:34.892464 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:03:35.054544 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:35.054497 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 09 04:03:35.058526 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:03:35.058499 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod592a60a4_9e18_4329_9f82_ddd3fd72c8a1.slice/crio-1480640ed5f5bb21d3175dba1aa5e7bd0dbaeb37f89f5e028ccbd80fedf8b34e WatchSource:0}: Error finding container 1480640ed5f5bb21d3175dba1aa5e7bd0dbaeb37f89f5e028ccbd80fedf8b34e: Status 404 returned error can't find the container with id 1480640ed5f5bb21d3175dba1aa5e7bd0dbaeb37f89f5e028ccbd80fedf8b34e Jul 09 04:03:35.520517 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:35.520481 2576 generic.go:356] "Generic (PLEG): container finished" podID="592a60a4-9e18-4329-9f82-ddd3fd72c8a1" containerID="ed23e8c1ebae2dc2c3c99e30a602f021e6b3f7a3d066df0394a8c7e36f183cee" exitCode=0 Jul 09 04:03:35.520998 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:35.520524 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"592a60a4-9e18-4329-9f82-ddd3fd72c8a1","Type":"ContainerDied","Data":"ed23e8c1ebae2dc2c3c99e30a602f021e6b3f7a3d066df0394a8c7e36f183cee"} Jul 09 04:03:35.520998 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:35.520589 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"592a60a4-9e18-4329-9f82-ddd3fd72c8a1","Type":"ContainerStarted","Data":"1480640ed5f5bb21d3175dba1aa5e7bd0dbaeb37f89f5e028ccbd80fedf8b34e"} Jul 09 04:03:35.623577 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:35.623538 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b49aa39d-c297-487e-8752-580d258daa34" path="/var/lib/kubelet/pods/b49aa39d-c297-487e-8752-580d258daa34/volumes" Jul 09 04:03:36.532520 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:36.532484 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"592a60a4-9e18-4329-9f82-ddd3fd72c8a1","Type":"ContainerStarted","Data":"dc726d0ce40b8a30ac64054b55c3c33da0487225b7952dd0b201e21ea7005d5a"} Jul 09 04:03:36.532926 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:36.532581 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"592a60a4-9e18-4329-9f82-ddd3fd72c8a1","Type":"ContainerStarted","Data":"7e992380cb38ea3f27a6d1051b6177b2d3161e2fc908ac7ea18039119bacfcb3"} Jul 09 04:03:36.532926 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:36.532609 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"592a60a4-9e18-4329-9f82-ddd3fd72c8a1","Type":"ContainerStarted","Data":"32e07f1e24d3dafac5245cd72172c2bea859f5f06e9f7824512f739ed3e5330e"} Jul 09 04:03:36.532926 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:36.532633 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"592a60a4-9e18-4329-9f82-ddd3fd72c8a1","Type":"ContainerStarted","Data":"f26d7dddc4bc543a18dff83bacea479eabc1c18b46dc9c7aa67dc5cd01189d42"} Jul 09 04:03:36.532926 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:36.532655 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"592a60a4-9e18-4329-9f82-ddd3fd72c8a1","Type":"ContainerStarted","Data":"4773a23b1bb1cd2f336bfd4adb85458fa69889f44ff287de8a02ac685092c9b0"} Jul 09 04:03:36.532926 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:36.532678 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"592a60a4-9e18-4329-9f82-ddd3fd72c8a1","Type":"ContainerStarted","Data":"4308ace55676abae73ce7b6d7b152eb4b66a0a6112f48a098c3afba825b1730d"} Jul 09 04:03:36.584834 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:36.584781 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.565699667 podStartE2EDuration="2.584762673s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.063006ms" isStatefulPod=false podCreationTimestamp="2026-07-09 04:03:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:03:36.58279009 +0000 UTC m=+225.458707680" watchObservedRunningTime="2026-07-09 04:03:36.584762673 +0000 UTC m=+225.460680261" Jul 09 04:03:39.892628 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:03:39.892595 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:04:34.893121 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:04:34.893074 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:04:34.908464 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:04:34.908433 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:04:35.760541 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:04:35.760512 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 09 04:04:51.469198 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:04:51.469169 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:04:51.477217 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:04:51.477192 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:04:51.479591 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:04:51.479565 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:04:51.482139 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:04:51.482113 2576 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 04:04:51.487310 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:04:51.487293 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:06:56.216037 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.216002 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs"] Jul 09 04:06:56.219143 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.219127 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.222063 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.222028 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 04:06:56.222195 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.222028 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 04:06:56.222597 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.222567 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 04:06:56.223482 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.223279 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 04:06:56.225471 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.225451 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7rq67\"" Jul 09 04:06:56.239537 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.239507 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs"] Jul 09 04:06:56.306941 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.306895 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/63d0fc88-2766-4c33-9a26-6f310339ecd2-webhook-cert\") pod \"opendatahub-operator-controller-manager-df844f965-bdbfs\" (UID: \"63d0fc88-2766-4c33-9a26-6f310339ecd2\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.307127 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.306970 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63d0fc88-2766-4c33-9a26-6f310339ecd2-tmp\") pod \"opendatahub-operator-controller-manager-df844f965-bdbfs\" (UID: \"63d0fc88-2766-4c33-9a26-6f310339ecd2\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.307127 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.306993 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/63d0fc88-2766-4c33-9a26-6f310339ecd2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-df844f965-bdbfs\" (UID: \"63d0fc88-2766-4c33-9a26-6f310339ecd2\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.307127 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.307024 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/63d0fc88-2766-4c33-9a26-6f310339ecd2-manifests\") pod \"opendatahub-operator-controller-manager-df844f965-bdbfs\" (UID: \"63d0fc88-2766-4c33-9a26-6f310339ecd2\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.307127 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.307060 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mqsq\" (UniqueName: \"kubernetes.io/projected/63d0fc88-2766-4c33-9a26-6f310339ecd2-kube-api-access-8mqsq\") pod \"opendatahub-operator-controller-manager-df844f965-bdbfs\" (UID: \"63d0fc88-2766-4c33-9a26-6f310339ecd2\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.408042 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.407997 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/63d0fc88-2766-4c33-9a26-6f310339ecd2-webhook-cert\") pod \"opendatahub-operator-controller-manager-df844f965-bdbfs\" (UID: \"63d0fc88-2766-4c33-9a26-6f310339ecd2\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.408253 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.408072 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63d0fc88-2766-4c33-9a26-6f310339ecd2-tmp\") pod \"opendatahub-operator-controller-manager-df844f965-bdbfs\" (UID: \"63d0fc88-2766-4c33-9a26-6f310339ecd2\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.408253 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.408101 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/63d0fc88-2766-4c33-9a26-6f310339ecd2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-df844f965-bdbfs\" (UID: \"63d0fc88-2766-4c33-9a26-6f310339ecd2\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.408253 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.408131 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/63d0fc88-2766-4c33-9a26-6f310339ecd2-manifests\") pod \"opendatahub-operator-controller-manager-df844f965-bdbfs\" (UID: \"63d0fc88-2766-4c33-9a26-6f310339ecd2\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.408407 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.408288 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mqsq\" (UniqueName: \"kubernetes.io/projected/63d0fc88-2766-4c33-9a26-6f310339ecd2-kube-api-access-8mqsq\") pod \"opendatahub-operator-controller-manager-df844f965-bdbfs\" (UID: \"63d0fc88-2766-4c33-9a26-6f310339ecd2\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.408550 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.408523 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/63d0fc88-2766-4c33-9a26-6f310339ecd2-manifests\") pod \"opendatahub-operator-controller-manager-df844f965-bdbfs\" (UID: \"63d0fc88-2766-4c33-9a26-6f310339ecd2\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.410452 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.410430 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63d0fc88-2766-4c33-9a26-6f310339ecd2-tmp\") pod \"opendatahub-operator-controller-manager-df844f965-bdbfs\" (UID: \"63d0fc88-2766-4c33-9a26-6f310339ecd2\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.410566 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.410526 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/63d0fc88-2766-4c33-9a26-6f310339ecd2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-df844f965-bdbfs\" (UID: \"63d0fc88-2766-4c33-9a26-6f310339ecd2\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.410566 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.410552 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/63d0fc88-2766-4c33-9a26-6f310339ecd2-webhook-cert\") pod \"opendatahub-operator-controller-manager-df844f965-bdbfs\" (UID: \"63d0fc88-2766-4c33-9a26-6f310339ecd2\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.424522 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.424489 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mqsq\" (UniqueName: \"kubernetes.io/projected/63d0fc88-2766-4c33-9a26-6f310339ecd2-kube-api-access-8mqsq\") pod \"opendatahub-operator-controller-manager-df844f965-bdbfs\" (UID: \"63d0fc88-2766-4c33-9a26-6f310339ecd2\") " pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.530905 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.530793 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:06:56.678753 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.678713 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs"] Jul 09 04:06:56.681688 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:06:56.681654 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63d0fc88_2766_4c33_9a26_6f310339ecd2.slice/crio-49fe68ad3b229de9e231f8d9c8f8f0046ed820ac5db6b6dd0b5685b17471cc0c WatchSource:0}: Error finding container 49fe68ad3b229de9e231f8d9c8f8f0046ed820ac5db6b6dd0b5685b17471cc0c: Status 404 returned error can't find the container with id 49fe68ad3b229de9e231f8d9c8f8f0046ed820ac5db6b6dd0b5685b17471cc0c Jul 09 04:06:56.683433 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:56.683417 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 04:06:57.256209 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:06:57.256163 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" event={"ID":"63d0fc88-2766-4c33-9a26-6f310339ecd2","Type":"ContainerStarted","Data":"49fe68ad3b229de9e231f8d9c8f8f0046ed820ac5db6b6dd0b5685b17471cc0c"} Jul 09 04:07:00.268674 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:00.268633 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" event={"ID":"63d0fc88-2766-4c33-9a26-6f310339ecd2","Type":"ContainerStarted","Data":"95354adfa1c43ba88bf57370fb8868280093eb5008592479130532c3dd7e99a8"} Jul 09 04:07:01.273813 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:01.273773 2576 generic.go:356] "Generic (PLEG): container finished" podID="63d0fc88-2766-4c33-9a26-6f310339ecd2" containerID="95354adfa1c43ba88bf57370fb8868280093eb5008592479130532c3dd7e99a8" exitCode=0 Jul 09 04:07:01.274324 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:01.273882 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" event={"ID":"63d0fc88-2766-4c33-9a26-6f310339ecd2","Type":"ContainerDied","Data":"95354adfa1c43ba88bf57370fb8868280093eb5008592479130532c3dd7e99a8"} Jul 09 04:07:02.279768 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:02.279732 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" event={"ID":"63d0fc88-2766-4c33-9a26-6f310339ecd2","Type":"ContainerStarted","Data":"fe84f3b64239de8ffc1de1972d9785ad1b3da369121f2134fc9b5a728b23c84d"} Jul 09 04:07:02.280177 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:02.279818 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:07:02.302895 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:02.302823 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" podStartSLOduration=1.772962843 podStartE2EDuration="6.302808177s" totalImagesPullingTime="3.493461773s" totalInitContainerRuntime="1.036383561s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:06:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:02.301824306 +0000 UTC m=+431.177741896" watchObservedRunningTime="2026-07-09 04:07:02.302808177 +0000 UTC m=+431.178725766" Jul 09 04:07:13.285495 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:13.285406 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-df844f965-bdbfs" Jul 09 04:07:23.594516 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.594477 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx"] Jul 09 04:07:23.597662 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.597633 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" Jul 09 04:07:23.600527 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.600500 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 09 04:07:23.600651 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.600500 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 09 04:07:23.601670 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.601655 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 04:07:23.601773 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.601755 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 09 04:07:23.601891 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.601772 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 04:07:23.601955 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.601886 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ct2c6\"" Jul 09 04:07:23.608071 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.608050 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx"] Jul 09 04:07:23.736263 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.736224 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8862ca9d-20f9-4c65-96e6-aebbe90d95be-metrics-cert\") pod \"lws-controller-manager-765cd6b57c-mc2fx\" (UID: \"8862ca9d-20f9-4c65-96e6-aebbe90d95be\") " pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" Jul 09 04:07:23.736446 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.736276 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8862ca9d-20f9-4c65-96e6-aebbe90d95be-cert\") pod \"lws-controller-manager-765cd6b57c-mc2fx\" (UID: \"8862ca9d-20f9-4c65-96e6-aebbe90d95be\") " pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" Jul 09 04:07:23.736446 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.736352 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtzbd\" (UniqueName: \"kubernetes.io/projected/8862ca9d-20f9-4c65-96e6-aebbe90d95be-kube-api-access-gtzbd\") pod \"lws-controller-manager-765cd6b57c-mc2fx\" (UID: \"8862ca9d-20f9-4c65-96e6-aebbe90d95be\") " pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" Jul 09 04:07:23.736517 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.736436 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8862ca9d-20f9-4c65-96e6-aebbe90d95be-manager-config\") pod \"lws-controller-manager-765cd6b57c-mc2fx\" (UID: \"8862ca9d-20f9-4c65-96e6-aebbe90d95be\") " pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" Jul 09 04:07:23.837754 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.837719 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtzbd\" (UniqueName: \"kubernetes.io/projected/8862ca9d-20f9-4c65-96e6-aebbe90d95be-kube-api-access-gtzbd\") pod \"lws-controller-manager-765cd6b57c-mc2fx\" (UID: \"8862ca9d-20f9-4c65-96e6-aebbe90d95be\") " pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" Jul 09 04:07:23.837966 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.837767 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8862ca9d-20f9-4c65-96e6-aebbe90d95be-manager-config\") pod \"lws-controller-manager-765cd6b57c-mc2fx\" (UID: \"8862ca9d-20f9-4c65-96e6-aebbe90d95be\") " pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" Jul 09 04:07:23.837966 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.837799 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8862ca9d-20f9-4c65-96e6-aebbe90d95be-metrics-cert\") pod \"lws-controller-manager-765cd6b57c-mc2fx\" (UID: \"8862ca9d-20f9-4c65-96e6-aebbe90d95be\") " pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" Jul 09 04:07:23.837966 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.837857 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8862ca9d-20f9-4c65-96e6-aebbe90d95be-cert\") pod \"lws-controller-manager-765cd6b57c-mc2fx\" (UID: \"8862ca9d-20f9-4c65-96e6-aebbe90d95be\") " pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" Jul 09 04:07:23.838563 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.838529 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8862ca9d-20f9-4c65-96e6-aebbe90d95be-manager-config\") pod \"lws-controller-manager-765cd6b57c-mc2fx\" (UID: \"8862ca9d-20f9-4c65-96e6-aebbe90d95be\") " pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" Jul 09 04:07:23.840424 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.840389 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8862ca9d-20f9-4c65-96e6-aebbe90d95be-cert\") pod \"lws-controller-manager-765cd6b57c-mc2fx\" (UID: \"8862ca9d-20f9-4c65-96e6-aebbe90d95be\") " pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" Jul 09 04:07:23.840526 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.840449 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8862ca9d-20f9-4c65-96e6-aebbe90d95be-metrics-cert\") pod \"lws-controller-manager-765cd6b57c-mc2fx\" (UID: \"8862ca9d-20f9-4c65-96e6-aebbe90d95be\") " pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" Jul 09 04:07:23.851161 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.851086 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtzbd\" (UniqueName: \"kubernetes.io/projected/8862ca9d-20f9-4c65-96e6-aebbe90d95be-kube-api-access-gtzbd\") pod \"lws-controller-manager-765cd6b57c-mc2fx\" (UID: \"8862ca9d-20f9-4c65-96e6-aebbe90d95be\") " pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" Jul 09 04:07:23.907284 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:23.907244 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" Jul 09 04:07:24.049862 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.049786 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx"] Jul 09 04:07:24.050103 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:07:24.050075 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8862ca9d_20f9_4c65_96e6_aebbe90d95be.slice/crio-614213aa815e20313d7c519d528fecac6c278772a7eb8f03e5964cd7d6842953 WatchSource:0}: Error finding container 614213aa815e20313d7c519d528fecac6c278772a7eb8f03e5964cd7d6842953: Status 404 returned error can't find the container with id 614213aa815e20313d7c519d528fecac6c278772a7eb8f03e5964cd7d6842953 Jul 09 04:07:24.328896 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.328854 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5"] Jul 09 04:07:24.333405 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.333384 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.336509 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.336484 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-qkk4c\"" Jul 09 04:07:24.336648 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.336571 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 04:07:24.336648 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.336594 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 04:07:24.336648 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.336606 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 04:07:24.347501 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.347468 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5"] Jul 09 04:07:24.363090 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.363057 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" event={"ID":"8862ca9d-20f9-4c65-96e6-aebbe90d95be","Type":"ContainerStarted","Data":"614213aa815e20313d7c519d528fecac6c278772a7eb8f03e5964cd7d6842953"} Jul 09 04:07:24.422801 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.422765 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm"] Jul 09 04:07:24.425826 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.425810 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.428758 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.428735 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 09 04:07:24.428873 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.428810 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 09 04:07:24.429274 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.429258 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 09 04:07:24.429484 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.429463 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-xrr7n\"" Jul 09 04:07:24.439090 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.439065 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm"] Jul 09 04:07:24.442087 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.442060 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.442194 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.442116 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsdjw\" (UniqueName: \"kubernetes.io/projected/a15fa495-0dc3-494b-86ab-d6646f5de7bf-kube-api-access-rsdjw\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.442194 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.442183 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.442269 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.442208 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.442269 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.442247 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.442331 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.442272 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.442331 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.442298 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.442394 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.442346 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-data\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.442394 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.442372 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-token\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.543156 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543123 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/75bb459b-3eac-4cdd-be45-55d5d3affc0c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.543156 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543168 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/75bb459b-3eac-4cdd-be45-55d5d3affc0c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.543395 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543192 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.543395 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543267 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/75bb459b-3eac-4cdd-be45-55d5d3affc0c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.543395 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543308 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsdjw\" (UniqueName: \"kubernetes.io/projected/a15fa495-0dc3-494b-86ab-d6646f5de7bf-kube-api-access-rsdjw\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.543395 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543328 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/75bb459b-3eac-4cdd-be45-55d5d3affc0c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.543395 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543357 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/75bb459b-3eac-4cdd-be45-55d5d3affc0c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.543395 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543391 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75bb459b-3eac-4cdd-be45-55d5d3affc0c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.543668 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543446 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.543668 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543489 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.543668 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543538 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.543668 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543538 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.543668 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543575 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.543668 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543602 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.543668 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543632 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdfdj\" (UniqueName: \"kubernetes.io/projected/75bb459b-3eac-4cdd-be45-55d5d3affc0c-kube-api-access-wdfdj\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.543982 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543693 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-data\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.543982 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543717 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-token\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.543982 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543760 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.543982 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.543813 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.544148 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.544126 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-data\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.544283 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.544268 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.546033 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.546010 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.546588 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.546571 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.559514 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.559468 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-token\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.559666 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.559560 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsdjw\" (UniqueName: \"kubernetes.io/projected/a15fa495-0dc3-494b-86ab-d6646f5de7bf-kube-api-access-rsdjw\") pod \"data-science-gateway-data-science-gateway-class-548557588-zpkf5\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.644363 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.644331 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/75bb459b-3eac-4cdd-be45-55d5d3affc0c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.644820 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.644389 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/75bb459b-3eac-4cdd-be45-55d5d3affc0c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.644820 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.644416 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/75bb459b-3eac-4cdd-be45-55d5d3affc0c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.644820 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.644438 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/75bb459b-3eac-4cdd-be45-55d5d3affc0c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.644820 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.644471 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/75bb459b-3eac-4cdd-be45-55d5d3affc0c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.644820 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.644498 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:24.644820 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.644501 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75bb459b-3eac-4cdd-be45-55d5d3affc0c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.645172 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.644892 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdfdj\" (UniqueName: \"kubernetes.io/projected/75bb459b-3eac-4cdd-be45-55d5d3affc0c-kube-api-access-wdfdj\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.645294 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.645271 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/75bb459b-3eac-4cdd-be45-55d5d3affc0c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.646958 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.646933 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/75bb459b-3eac-4cdd-be45-55d5d3affc0c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.647064 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.647047 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/75bb459b-3eac-4cdd-be45-55d5d3affc0c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.647271 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.647243 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/75bb459b-3eac-4cdd-be45-55d5d3affc0c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.647310 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.647251 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/75bb459b-3eac-4cdd-be45-55d5d3affc0c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.655733 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.655703 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75bb459b-3eac-4cdd-be45-55d5d3affc0c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.655831 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.655806 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdfdj\" (UniqueName: \"kubernetes.io/projected/75bb459b-3eac-4cdd-be45-55d5d3affc0c-kube-api-access-wdfdj\") pod \"istiod-openshift-gateway-94bb8fbfd-j4xcm\" (UID: \"75bb459b-3eac-4cdd-be45-55d5d3affc0c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.735492 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.735380 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:24.836411 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.831091 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5"] Jul 09 04:07:24.836411 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:07:24.834168 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda15fa495_0dc3_494b_86ab_d6646f5de7bf.slice/crio-a51217b7accfc8f9f4671f4ce7b098de483003ebf42311036d5d9fe03a59690e WatchSource:0}: Error finding container a51217b7accfc8f9f4671f4ce7b098de483003ebf42311036d5d9fe03a59690e: Status 404 returned error can't find the container with id a51217b7accfc8f9f4671f4ce7b098de483003ebf42311036d5d9fe03a59690e Jul 09 04:07:24.928492 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:24.928404 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm"] Jul 09 04:07:24.933428 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:07:24.933393 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75bb459b_3eac_4cdd_be45_55d5d3affc0c.slice/crio-61ff06709419fbc09557d86f6823cd5838ddac279fa71df4890cc8ff8dac4ab0 WatchSource:0}: Error finding container 61ff06709419fbc09557d86f6823cd5838ddac279fa71df4890cc8ff8dac4ab0: Status 404 returned error can't find the container with id 61ff06709419fbc09557d86f6823cd5838ddac279fa71df4890cc8ff8dac4ab0 Jul 09 04:07:25.370203 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:25.370108 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" event={"ID":"a15fa495-0dc3-494b-86ab-d6646f5de7bf","Type":"ContainerStarted","Data":"a51217b7accfc8f9f4671f4ce7b098de483003ebf42311036d5d9fe03a59690e"} Jul 09 04:07:25.372041 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:25.372007 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" event={"ID":"75bb459b-3eac-4cdd-be45-55d5d3affc0c","Type":"ContainerStarted","Data":"61ff06709419fbc09557d86f6823cd5838ddac279fa71df4890cc8ff8dac4ab0"} Jul 09 04:07:29.105091 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:29.105052 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5"] Jul 09 04:07:29.110216 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:29.110167 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:07:29.110330 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:29.110261 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:07:29.113340 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:29.113311 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:07:29.113429 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:29.113374 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:07:29.113429 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:29.113402 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:07:29.394729 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:29.394689 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" event={"ID":"8862ca9d-20f9-4c65-96e6-aebbe90d95be","Type":"ContainerStarted","Data":"df27265385384f46e0ae22dae78ff20f563998e3b2b262acd498b2ee619f52d1"} Jul 09 04:07:29.394954 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:29.394824 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" Jul 09 04:07:29.396511 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:29.396485 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" event={"ID":"a15fa495-0dc3-494b-86ab-d6646f5de7bf","Type":"ContainerStarted","Data":"501ddc5455d3fce8d1095cacfade2d33c248d16137fc4a6d8e28370171f47881"} Jul 09 04:07:29.396655 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:29.396563 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" podUID="a15fa495-0dc3-494b-86ab-d6646f5de7bf" containerName="istio-proxy" containerID="cri-o://501ddc5455d3fce8d1095cacfade2d33c248d16137fc4a6d8e28370171f47881" gracePeriod=30 Jul 09 04:07:29.398317 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:29.398291 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" event={"ID":"75bb459b-3eac-4cdd-be45-55d5d3affc0c","Type":"ContainerStarted","Data":"2c0925e2b9cd7f4b38c1419729aa89109fe8936b2f0fd1b004b5913844a79148"} Jul 09 04:07:29.398456 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:29.398404 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:29.413626 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:29.413567 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" podStartSLOduration=1.359160691 podStartE2EDuration="6.413544905s" totalImagesPullingTime="5.054384214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:29.411249097 +0000 UTC m=+458.287166684" watchObservedRunningTime="2026-07-09 04:07:29.413544905 +0000 UTC m=+458.289462494" Jul 09 04:07:29.430020 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:29.429961 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" podStartSLOduration=1.255936805 podStartE2EDuration="5.42992796s" totalImagesPullingTime="4.173991155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:29.428651092 +0000 UTC m=+458.304568684" watchObservedRunningTime="2026-07-09 04:07:29.42992796 +0000 UTC m=+458.305845547" Jul 09 04:07:29.448720 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:29.448667 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" podStartSLOduration=1.17348009 podStartE2EDuration="5.448650409s" totalImagesPullingTime="4.275170319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:29.44689956 +0000 UTC m=+458.322817149" watchObservedRunningTime="2026-07-09 04:07:29.448650409 +0000 UTC m=+458.324567997" Jul 09 04:07:30.404180 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:30.404155 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-j4xcm" Jul 09 04:07:34.659557 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.659491 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:34.843861 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.843796 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-workload-certs\") pod \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " Jul 09 04:07:34.844055 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.843877 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istiod-ca-cert\") pod \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " Jul 09 04:07:34.844055 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.843906 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-credential-socket\") pod \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " Jul 09 04:07:34.844055 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.843964 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-token\") pod \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " Jul 09 04:07:34.844055 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.844047 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-podinfo\") pod \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " Jul 09 04:07:34.844250 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.844075 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-workload-socket\") pod \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " Jul 09 04:07:34.844250 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.844100 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-envoy\") pod \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " Jul 09 04:07:34.844250 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.844140 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a15fa495-0dc3-494b-86ab-d6646f5de7bf" (UID: "a15fa495-0dc3-494b-86ab-d6646f5de7bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:34.844250 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.844210 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsdjw\" (UniqueName: \"kubernetes.io/projected/a15fa495-0dc3-494b-86ab-d6646f5de7bf-kube-api-access-rsdjw\") pod \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " Jul 09 04:07:34.844250 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.844241 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-data\") pod \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\" (UID: \"a15fa495-0dc3-494b-86ab-d6646f5de7bf\") " Jul 09 04:07:34.844500 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.844347 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a15fa495-0dc3-494b-86ab-d6646f5de7bf" (UID: "a15fa495-0dc3-494b-86ab-d6646f5de7bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:34.844500 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.844352 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a15fa495-0dc3-494b-86ab-d6646f5de7bf" (UID: "a15fa495-0dc3-494b-86ab-d6646f5de7bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:07:34.844500 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.844496 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-workload-socket\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:07:34.844659 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.844514 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-workload-certs\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:07:34.844659 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.844526 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istiod-ca-cert\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:07:34.844659 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.844535 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-data" (OuterVolumeSpecName: "istio-data") pod "a15fa495-0dc3-494b-86ab-d6646f5de7bf" (UID: "a15fa495-0dc3-494b-86ab-d6646f5de7bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:34.844659 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.844638 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a15fa495-0dc3-494b-86ab-d6646f5de7bf" (UID: "a15fa495-0dc3-494b-86ab-d6646f5de7bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:34.846355 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.846335 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a15fa495-0dc3-494b-86ab-d6646f5de7bf" (UID: "a15fa495-0dc3-494b-86ab-d6646f5de7bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:07:34.846427 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.846372 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-token" (OuterVolumeSpecName: "istio-token") pod "a15fa495-0dc3-494b-86ab-d6646f5de7bf" (UID: "a15fa495-0dc3-494b-86ab-d6646f5de7bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:07:34.846591 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.846564 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a15fa495-0dc3-494b-86ab-d6646f5de7bf" (UID: "a15fa495-0dc3-494b-86ab-d6646f5de7bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:07:34.846692 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.846565 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a15fa495-0dc3-494b-86ab-d6646f5de7bf-kube-api-access-rsdjw" (OuterVolumeSpecName: "kube-api-access-rsdjw") pod "a15fa495-0dc3-494b-86ab-d6646f5de7bf" (UID: "a15fa495-0dc3-494b-86ab-d6646f5de7bf"). InnerVolumeSpecName "kube-api-access-rsdjw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:07:34.945394 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.945295 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-podinfo\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:07:34.945394 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.945327 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-envoy\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:07:34.945394 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.945336 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsdjw\" (UniqueName: \"kubernetes.io/projected/a15fa495-0dc3-494b-86ab-d6646f5de7bf-kube-api-access-rsdjw\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:07:34.945394 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.945345 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-data\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:07:34.945394 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.945355 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a15fa495-0dc3-494b-86ab-d6646f5de7bf-credential-socket\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:07:34.945394 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:34.945365 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a15fa495-0dc3-494b-86ab-d6646f5de7bf-istio-token\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:07:35.423707 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:35.423668 2576 generic.go:356] "Generic (PLEG): container finished" podID="a15fa495-0dc3-494b-86ab-d6646f5de7bf" containerID="501ddc5455d3fce8d1095cacfade2d33c248d16137fc4a6d8e28370171f47881" exitCode=0 Jul 09 04:07:35.423881 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:35.423722 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" Jul 09 04:07:35.423881 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:35.423755 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" event={"ID":"a15fa495-0dc3-494b-86ab-d6646f5de7bf","Type":"ContainerDied","Data":"501ddc5455d3fce8d1095cacfade2d33c248d16137fc4a6d8e28370171f47881"} Jul 09 04:07:35.423881 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:35.423813 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5" event={"ID":"a15fa495-0dc3-494b-86ab-d6646f5de7bf","Type":"ContainerDied","Data":"a51217b7accfc8f9f4671f4ce7b098de483003ebf42311036d5d9fe03a59690e"} Jul 09 04:07:35.423881 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:35.423864 2576 scope.go:117] "RemoveContainer" containerID="501ddc5455d3fce8d1095cacfade2d33c248d16137fc4a6d8e28370171f47881" Jul 09 04:07:35.433637 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:35.433621 2576 scope.go:117] "RemoveContainer" containerID="501ddc5455d3fce8d1095cacfade2d33c248d16137fc4a6d8e28370171f47881" Jul 09 04:07:35.434417 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:07:35.434245 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"501ddc5455d3fce8d1095cacfade2d33c248d16137fc4a6d8e28370171f47881\": container with ID starting with 501ddc5455d3fce8d1095cacfade2d33c248d16137fc4a6d8e28370171f47881 not found: ID does not exist" containerID="501ddc5455d3fce8d1095cacfade2d33c248d16137fc4a6d8e28370171f47881" Jul 09 04:07:35.434417 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:35.434281 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"501ddc5455d3fce8d1095cacfade2d33c248d16137fc4a6d8e28370171f47881"} err="failed to get container status \"501ddc5455d3fce8d1095cacfade2d33c248d16137fc4a6d8e28370171f47881\": rpc error: code = NotFound desc = could not find container \"501ddc5455d3fce8d1095cacfade2d33c248d16137fc4a6d8e28370171f47881\": container with ID starting with 501ddc5455d3fce8d1095cacfade2d33c248d16137fc4a6d8e28370171f47881 not found: ID does not exist" Jul 09 04:07:35.445034 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:35.444993 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5"] Jul 09 04:07:35.450980 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:35.450950 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-zpkf5"] Jul 09 04:07:35.623100 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:35.623070 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a15fa495-0dc3-494b-86ab-d6646f5de7bf" path="/var/lib/kubelet/pods/a15fa495-0dc3-494b-86ab-d6646f5de7bf/volumes" Jul 09 04:07:40.405035 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:07:40.405004 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-765cd6b57c-mc2fx" Jul 09 04:09:45.038561 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.038524 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz"] Jul 09 04:09:45.038986 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.038827 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a15fa495-0dc3-494b-86ab-d6646f5de7bf" containerName="istio-proxy" Jul 09 04:09:45.038986 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.038854 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a15fa495-0dc3-494b-86ab-d6646f5de7bf" containerName="istio-proxy" Jul 09 04:09:45.038986 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.038910 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a15fa495-0dc3-494b-86ab-d6646f5de7bf" containerName="istio-proxy" Jul 09 04:09:45.041783 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.041757 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.044685 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.044666 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-7kh7v\"" Jul 09 04:09:45.053540 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.053511 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz"] Jul 09 04:09:45.129700 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.129661 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ef6eb95-116d-41ea-b5a6-44159057e047-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.129700 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.129714 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef6eb95-116d-41ea-b5a6-44159057e047-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.129996 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.129814 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ef6eb95-116d-41ea-b5a6-44159057e047-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.129996 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.129905 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ef6eb95-116d-41ea-b5a6-44159057e047-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.129996 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.129940 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ef6eb95-116d-41ea-b5a6-44159057e047-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.129996 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.129964 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ef6eb95-116d-41ea-b5a6-44159057e047-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.130138 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.130010 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef6eb95-116d-41ea-b5a6-44159057e047-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.130138 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.130055 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ef6eb95-116d-41ea-b5a6-44159057e047-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.130138 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.130097 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gkjz\" (UniqueName: \"kubernetes.io/projected/3ef6eb95-116d-41ea-b5a6-44159057e047-kube-api-access-5gkjz\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.231085 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.231045 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gkjz\" (UniqueName: \"kubernetes.io/projected/3ef6eb95-116d-41ea-b5a6-44159057e047-kube-api-access-5gkjz\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.231085 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.231081 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ef6eb95-116d-41ea-b5a6-44159057e047-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.231350 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.231105 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef6eb95-116d-41ea-b5a6-44159057e047-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.231350 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.231141 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ef6eb95-116d-41ea-b5a6-44159057e047-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.231350 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.231184 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ef6eb95-116d-41ea-b5a6-44159057e047-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.231350 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.231215 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ef6eb95-116d-41ea-b5a6-44159057e047-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.231350 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.231239 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ef6eb95-116d-41ea-b5a6-44159057e047-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.231350 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.231275 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef6eb95-116d-41ea-b5a6-44159057e047-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.231350 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.231317 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ef6eb95-116d-41ea-b5a6-44159057e047-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.231678 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.231549 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef6eb95-116d-41ea-b5a6-44159057e047-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.231760 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.231738 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ef6eb95-116d-41ea-b5a6-44159057e047-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.231867 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.231831 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ef6eb95-116d-41ea-b5a6-44159057e047-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.231940 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.231902 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ef6eb95-116d-41ea-b5a6-44159057e047-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.232131 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.232109 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ef6eb95-116d-41ea-b5a6-44159057e047-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.233993 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.233968 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ef6eb95-116d-41ea-b5a6-44159057e047-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.234151 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.234133 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ef6eb95-116d-41ea-b5a6-44159057e047-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.241375 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.241345 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ef6eb95-116d-41ea-b5a6-44159057e047-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.241491 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.241466 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gkjz\" (UniqueName: \"kubernetes.io/projected/3ef6eb95-116d-41ea-b5a6-44159057e047-kube-api-access-5gkjz\") pod \"maas-default-gateway-openshift-default-5896d4dd49-tqvrz\" (UID: \"3ef6eb95-116d-41ea-b5a6-44159057e047\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.355173 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.355075 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:45.493941 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.493891 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz"] Jul 09 04:09:45.496123 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:09:45.496090 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ef6eb95_116d_41ea_b5a6_44159057e047.slice/crio-e24c03b321952b3f3c2a3a296d1a8ee45133384d5aff70989b0abea2a5bbda9e WatchSource:0}: Error finding container e24c03b321952b3f3c2a3a296d1a8ee45133384d5aff70989b0abea2a5bbda9e: Status 404 returned error can't find the container with id e24c03b321952b3f3c2a3a296d1a8ee45133384d5aff70989b0abea2a5bbda9e Jul 09 04:09:45.498264 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.498229 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:09:45.498339 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.498313 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:09:45.498376 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.498356 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:09:45.945366 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.945326 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" event={"ID":"3ef6eb95-116d-41ea-b5a6-44159057e047","Type":"ContainerStarted","Data":"24ff79e0ce1586ced85ce010b2f038b8204b687317e49c18c39b90a1b1c6a48a"} Jul 09 04:09:45.945551 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.945379 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" event={"ID":"3ef6eb95-116d-41ea-b5a6-44159057e047","Type":"ContainerStarted","Data":"e24c03b321952b3f3c2a3a296d1a8ee45133384d5aff70989b0abea2a5bbda9e"} Jul 09 04:09:45.964788 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:45.964737 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" podStartSLOduration=0.964721412 podStartE2EDuration="964.721412ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:09:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:09:45.964038631 +0000 UTC m=+594.839956221" watchObservedRunningTime="2026-07-09 04:09:45.964721412 +0000 UTC m=+594.840638999" Jul 09 04:09:46.355599 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:46.355521 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:46.363617 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:46.363590 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:46.949148 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:46.949108 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:46.950204 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:46.950186 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:09:51.496731 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:51.496696 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:09:51.504181 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:51.504158 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:09:51.504307 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:51.504200 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:09:51.510864 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:09:51.510833 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:10:18.803734 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:18.803641 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-557xq"] Jul 09 04:10:18.806937 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:18.806920 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-557xq" Jul 09 04:10:18.809526 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:18.809506 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 04:10:18.809800 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:18.809782 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 04:10:18.809870 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:18.809821 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 09 04:10:18.810651 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:18.810636 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-hx48z\"" Jul 09 04:10:18.816538 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:18.816509 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-557xq"] Jul 09 04:10:18.919476 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:18.919429 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzdn7\" (UniqueName: \"kubernetes.io/projected/5792dbfd-9d80-4de8-a04a-552a6bf8afd5-kube-api-access-mzdn7\") pod \"postgres-69fb4594cf-557xq\" (UID: \"5792dbfd-9d80-4de8-a04a-552a6bf8afd5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-557xq" Jul 09 04:10:18.919661 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:18.919502 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5792dbfd-9d80-4de8-a04a-552a6bf8afd5-data\") pod \"postgres-69fb4594cf-557xq\" (UID: \"5792dbfd-9d80-4de8-a04a-552a6bf8afd5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-557xq" Jul 09 04:10:19.020088 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:19.020051 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzdn7\" (UniqueName: \"kubernetes.io/projected/5792dbfd-9d80-4de8-a04a-552a6bf8afd5-kube-api-access-mzdn7\") pod \"postgres-69fb4594cf-557xq\" (UID: \"5792dbfd-9d80-4de8-a04a-552a6bf8afd5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-557xq" Jul 09 04:10:19.020270 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:19.020120 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5792dbfd-9d80-4de8-a04a-552a6bf8afd5-data\") pod \"postgres-69fb4594cf-557xq\" (UID: \"5792dbfd-9d80-4de8-a04a-552a6bf8afd5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-557xq" Jul 09 04:10:19.020496 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:19.020477 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5792dbfd-9d80-4de8-a04a-552a6bf8afd5-data\") pod \"postgres-69fb4594cf-557xq\" (UID: \"5792dbfd-9d80-4de8-a04a-552a6bf8afd5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-557xq" Jul 09 04:10:19.028269 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:19.028234 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzdn7\" (UniqueName: \"kubernetes.io/projected/5792dbfd-9d80-4de8-a04a-552a6bf8afd5-kube-api-access-mzdn7\") pod \"postgres-69fb4594cf-557xq\" (UID: \"5792dbfd-9d80-4de8-a04a-552a6bf8afd5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-557xq" Jul 09 04:10:19.118474 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:19.118387 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-557xq" Jul 09 04:10:19.249206 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:19.249177 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-557xq"] Jul 09 04:10:19.251580 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:10:19.251545 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5792dbfd_9d80_4de8_a04a_552a6bf8afd5.slice/crio-98f93dc036bae95d2acf0210892b87fac16a1b4d396f313d6c8cfe1116ca31ec WatchSource:0}: Error finding container 98f93dc036bae95d2acf0210892b87fac16a1b4d396f313d6c8cfe1116ca31ec: Status 404 returned error can't find the container with id 98f93dc036bae95d2acf0210892b87fac16a1b4d396f313d6c8cfe1116ca31ec Jul 09 04:10:20.084006 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:20.083961 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-557xq" event={"ID":"5792dbfd-9d80-4de8-a04a-552a6bf8afd5","Type":"ContainerStarted","Data":"98f93dc036bae95d2acf0210892b87fac16a1b4d396f313d6c8cfe1116ca31ec"} Jul 09 04:10:24.103424 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:24.103332 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-557xq" event={"ID":"5792dbfd-9d80-4de8-a04a-552a6bf8afd5","Type":"ContainerStarted","Data":"11db4864a6cec5468507d5b0875e6ad13429b12d88458af5682a5502be1ad2ad"} Jul 09 04:10:24.103795 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:24.103440 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-557xq" Jul 09 04:10:24.119125 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:24.119059 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-557xq" podStartSLOduration=1.54370223 podStartE2EDuration="6.119039633s" totalImagesPullingTime="4.575337403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:10:24.117304922 +0000 UTC m=+632.993222511" watchObservedRunningTime="2026-07-09 04:10:24.119039633 +0000 UTC m=+632.994957220" Jul 09 04:10:30.136530 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:10:30.136500 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-557xq" Jul 09 04:11:17.992032 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:17.991990 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-tqvrz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.22:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 04:11:17.992450 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:17.992052 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" podUID="3ef6eb95-116d-41ea-b5a6-44159057e047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.22:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 04:11:19.336133 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:19.336046 2576 generic.go:356] "Generic (PLEG): container finished" podID="3ef6eb95-116d-41ea-b5a6-44159057e047" containerID="24ff79e0ce1586ced85ce010b2f038b8204b687317e49c18c39b90a1b1c6a48a" exitCode=0 Jul 09 04:11:19.336133 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:19.336084 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" event={"ID":"3ef6eb95-116d-41ea-b5a6-44159057e047","Type":"ContainerDied","Data":"24ff79e0ce1586ced85ce010b2f038b8204b687317e49c18c39b90a1b1c6a48a"} Jul 09 04:11:19.336678 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:19.336538 2576 scope.go:117] "RemoveContainer" containerID="24ff79e0ce1586ced85ce010b2f038b8204b687317e49c18c39b90a1b1c6a48a" Jul 09 04:11:19.337516 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:19.337489 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:11:19.337602 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:19.337552 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:11:19.337602 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:19.337584 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:11:20.345896 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:20.345834 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" event={"ID":"3ef6eb95-116d-41ea-b5a6-44159057e047","Type":"ContainerStarted","Data":"9dcc07785ab7261096093eda306f72daa6220596444d01ea355ca50118284680"} Jul 09 04:11:20.355473 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:20.355435 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:11:20.355724 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:20.355697 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-tqvrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.22:15021/healthz/ready\": dial tcp 10.132.0.22:15021: connect: connection refused" start-of-body= Jul 09 04:11:20.355796 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:20.355758 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" podUID="3ef6eb95-116d-41ea-b5a6-44159057e047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.22:15021/healthz/ready\": dial tcp 10.132.0.22:15021: connect: connection refused" Jul 09 04:11:21.356218 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:21.356192 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-tqvrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.22:15021/healthz/ready\": dial tcp 10.132.0.22:15021: connect: connection refused" start-of-body= Jul 09 04:11:21.356606 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:21.356238 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" podUID="3ef6eb95-116d-41ea-b5a6-44159057e047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.22:15021/healthz/ready\": dial tcp 10.132.0.22:15021: connect: connection refused" Jul 09 04:11:22.355782 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:22.355753 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-tqvrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.22:15021/healthz/ready\": dial tcp 10.132.0.22:15021: connect: connection refused" start-of-body= Jul 09 04:11:22.355986 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:22.355804 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" podUID="3ef6eb95-116d-41ea-b5a6-44159057e047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.22:15021/healthz/ready\": dial tcp 10.132.0.22:15021: connect: connection refused" Jul 09 04:11:23.355854 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:23.355811 2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-tqvrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.22:15021/healthz/ready\": dial tcp 10.132.0.22:15021: connect: connection refused" start-of-body= Jul 09 04:11:23.356236 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:23.355911 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" podUID="3ef6eb95-116d-41ea-b5a6-44159057e047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.22:15021/healthz/ready\": dial tcp 10.132.0.22:15021: connect: connection refused" Jul 09 04:11:24.359634 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:24.359606 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:11:24.360136 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:24.360025 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:11:24.360749 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:24.360729 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-tqvrz" Jul 09 04:11:31.298939 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.298904 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d"] Jul 09 04:11:31.302790 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.302771 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.306782 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.306756 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-trvcm\"" Jul 09 04:11:31.306782 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.306760 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 09 04:11:31.307004 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.306795 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 04:11:31.307004 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.306763 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 04:11:31.309993 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.309967 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d"] Jul 09 04:11:31.353586 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.353548 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.353586 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.353583 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.353830 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.353612 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.353830 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.353698 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.353830 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.353733 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.353830 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.353787 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db7sr\" (UniqueName: \"kubernetes.io/projected/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-kube-api-access-db7sr\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.454558 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.454524 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db7sr\" (UniqueName: \"kubernetes.io/projected/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-kube-api-access-db7sr\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.454802 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.454595 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.454802 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.454619 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.454802 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.454740 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.454802 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.454797 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.455052 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.454823 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.455374 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.455316 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.455374 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.455341 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.455516 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.455392 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.457267 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.457238 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.457393 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.457375 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.462534 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.462516 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-db7sr\" (UniqueName: \"kubernetes.io/projected/51b3bac2-2bc9-4d7d-847a-7e32af5946c2-kube-api-access-db7sr\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d\" (UID: \"51b3bac2-2bc9-4d7d-847a-7e32af5946c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.613884 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.613756 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:31.748606 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:31.748433 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d"] Jul 09 04:11:31.751235 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:11:31.751197 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51b3bac2_2bc9_4d7d_847a_7e32af5946c2.slice/crio-9bd998c4d668551e3cdeccbaa34c055df3e5feb4526b9c2115cf9089846e6501 WatchSource:0}: Error finding container 9bd998c4d668551e3cdeccbaa34c055df3e5feb4526b9c2115cf9089846e6501: Status 404 returned error can't find the container with id 9bd998c4d668551e3cdeccbaa34c055df3e5feb4526b9c2115cf9089846e6501 Jul 09 04:11:32.402163 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:32.402128 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" event={"ID":"51b3bac2-2bc9-4d7d-847a-7e32af5946c2","Type":"ContainerStarted","Data":"9bd998c4d668551e3cdeccbaa34c055df3e5feb4526b9c2115cf9089846e6501"} Jul 09 04:11:33.407776 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:33.407735 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" event={"ID":"51b3bac2-2bc9-4d7d-847a-7e32af5946c2","Type":"ContainerStarted","Data":"7a290bedf20fb5ce6b04102cdc4fda89fb5ec3bed8b8bb66d4a74be46747a5ae"} Jul 09 04:11:33.424872 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:33.424801 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" podStartSLOduration=1.4093237539999999 podStartE2EDuration="2.424784062s" totalImagesPullingTime="1.015460308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:33.424467422 +0000 UTC m=+702.300385010" watchObservedRunningTime="2026-07-09 04:11:33.424784062 +0000 UTC m=+702.300701650" Jul 09 04:11:41.614254 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:41.614206 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:41.614790 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:41.614272 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:41.627214 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:41.627189 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:42.455720 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:42.455690 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d" Jul 09 04:11:48.690918 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.690880 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4"] Jul 09 04:11:48.695988 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.695962 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.699785 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.699761 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4"] Jul 09 04:11:48.700737 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.700716 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 09 04:11:48.820497 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.820463 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3a16cdc-993c-49c5-9395-5335ca9c586d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.820697 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.820519 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3a16cdc-993c-49c5-9395-5335ca9c586d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.820697 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.820588 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3a16cdc-993c-49c5-9395-5335ca9c586d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.820697 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.820634 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw49h\" (UniqueName: \"kubernetes.io/projected/d3a16cdc-993c-49c5-9395-5335ca9c586d-kube-api-access-nw49h\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.820697 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.820685 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3a16cdc-993c-49c5-9395-5335ca9c586d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.820921 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.820719 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3a16cdc-993c-49c5-9395-5335ca9c586d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.922037 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.922001 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3a16cdc-993c-49c5-9395-5335ca9c586d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.922223 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.922062 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3a16cdc-993c-49c5-9395-5335ca9c586d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.922223 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.922078 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3a16cdc-993c-49c5-9395-5335ca9c586d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.922223 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.922103 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw49h\" (UniqueName: \"kubernetes.io/projected/d3a16cdc-993c-49c5-9395-5335ca9c586d-kube-api-access-nw49h\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.922223 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.922152 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3a16cdc-993c-49c5-9395-5335ca9c586d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.922223 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.922177 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3a16cdc-993c-49c5-9395-5335ca9c586d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.922569 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.922538 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3a16cdc-993c-49c5-9395-5335ca9c586d-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.922627 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.922578 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3a16cdc-993c-49c5-9395-5335ca9c586d-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.922627 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.922555 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3a16cdc-993c-49c5-9395-5335ca9c586d-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.924372 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.924350 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3a16cdc-993c-49c5-9395-5335ca9c586d-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.924482 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.924451 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3a16cdc-993c-49c5-9395-5335ca9c586d-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:48.930054 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:48.930036 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw49h\" (UniqueName: \"kubernetes.io/projected/d3a16cdc-993c-49c5-9395-5335ca9c586d-kube-api-access-nw49h\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lklv4\" (UID: \"d3a16cdc-993c-49c5-9395-5335ca9c586d\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:49.006480 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:49.006388 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:49.135741 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:49.135715 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4"] Jul 09 04:11:49.137744 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:11:49.137720 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3a16cdc_993c_49c5_9395_5335ca9c586d.slice/crio-145ee90221b342ef99689d8659038cf1c0fe1ad8cf3611e541c40e68a2127b38 WatchSource:0}: Error finding container 145ee90221b342ef99689d8659038cf1c0fe1ad8cf3611e541c40e68a2127b38: Status 404 returned error can't find the container with id 145ee90221b342ef99689d8659038cf1c0fe1ad8cf3611e541c40e68a2127b38 Jul 09 04:11:49.474152 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:49.474112 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" event={"ID":"d3a16cdc-993c-49c5-9395-5335ca9c586d","Type":"ContainerStarted","Data":"7e8cce7dc7a66f18ac4f70650b96d35222877517a71a50be34be4244161712d3"} Jul 09 04:11:49.474323 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:49.474173 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" event={"ID":"d3a16cdc-993c-49c5-9395-5335ca9c586d","Type":"ContainerStarted","Data":"145ee90221b342ef99689d8659038cf1c0fe1ad8cf3611e541c40e68a2127b38"} Jul 09 04:11:49.492017 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:49.491964 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" podStartSLOduration=1.306756124 podStartE2EDuration="1.491947855s" totalImagesPullingTime="185.191731ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:49.490192099 +0000 UTC m=+718.366109688" watchObservedRunningTime="2026-07-09 04:11:49.491947855 +0000 UTC m=+718.367865444" Jul 09 04:11:59.007446 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:59.007409 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:59.008025 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:59.007471 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:59.020008 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:59.019978 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:11:59.527391 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:11:59.527365 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lklv4" Jul 09 04:12:07.487548 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.487511 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh"] Jul 09 04:12:07.493412 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.493394 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.496182 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.496165 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 09 04:12:07.500621 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.500601 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh"] Jul 09 04:12:07.584907 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.584865 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb8mz\" (UniqueName: \"kubernetes.io/projected/50d77141-63fe-4146-a629-50b044a5aa0f-kube-api-access-gb8mz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.585048 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.584961 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50d77141-63fe-4146-a629-50b044a5aa0f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.585048 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.584982 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50d77141-63fe-4146-a629-50b044a5aa0f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.585048 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.584997 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50d77141-63fe-4146-a629-50b044a5aa0f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.585188 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.585053 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50d77141-63fe-4146-a629-50b044a5aa0f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.585188 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.585094 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50d77141-63fe-4146-a629-50b044a5aa0f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.686080 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.686044 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb8mz\" (UniqueName: \"kubernetes.io/projected/50d77141-63fe-4146-a629-50b044a5aa0f-kube-api-access-gb8mz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.686258 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.686100 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50d77141-63fe-4146-a629-50b044a5aa0f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.686258 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.686209 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50d77141-63fe-4146-a629-50b044a5aa0f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.686258 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.686242 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50d77141-63fe-4146-a629-50b044a5aa0f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.686383 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.686270 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50d77141-63fe-4146-a629-50b044a5aa0f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.686383 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.686309 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50d77141-63fe-4146-a629-50b044a5aa0f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.686615 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.686589 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50d77141-63fe-4146-a629-50b044a5aa0f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.686744 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.686651 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50d77141-63fe-4146-a629-50b044a5aa0f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.686744 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.686715 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50d77141-63fe-4146-a629-50b044a5aa0f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.688457 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.688432 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50d77141-63fe-4146-a629-50b044a5aa0f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.688616 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.688601 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50d77141-63fe-4146-a629-50b044a5aa0f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.694072 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.694046 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb8mz\" (UniqueName: \"kubernetes.io/projected/50d77141-63fe-4146-a629-50b044a5aa0f-kube-api-access-gb8mz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bm6fh\" (UID: \"50d77141-63fe-4146-a629-50b044a5aa0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.804450 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.804360 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:07.933948 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.933925 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh"] Jul 09 04:12:07.936095 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:12:07.936071 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d77141_63fe_4146_a629_50b044a5aa0f.slice/crio-0e7a2d362ae79aa1b76d47c07f4a99e4a6832545f303b2a500238297940a414b WatchSource:0}: Error finding container 0e7a2d362ae79aa1b76d47c07f4a99e4a6832545f303b2a500238297940a414b: Status 404 returned error can't find the container with id 0e7a2d362ae79aa1b76d47c07f4a99e4a6832545f303b2a500238297940a414b Jul 09 04:12:07.937373 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:07.937357 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 04:12:08.555090 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:08.554994 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" event={"ID":"50d77141-63fe-4146-a629-50b044a5aa0f","Type":"ContainerStarted","Data":"c4438765700d34221ded3f801c13bb0ad01688fb4c71b12786f3d63d28371305"} Jul 09 04:12:08.555090 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:08.555051 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" event={"ID":"50d77141-63fe-4146-a629-50b044a5aa0f","Type":"ContainerStarted","Data":"0e7a2d362ae79aa1b76d47c07f4a99e4a6832545f303b2a500238297940a414b"} Jul 09 04:12:08.574040 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:08.573989 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" podStartSLOduration=1.3591052160000001 podStartE2EDuration="1.573972426s" totalImagesPullingTime="214.86721ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:12:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:12:08.572475691 +0000 UTC m=+737.448393285" watchObservedRunningTime="2026-07-09 04:12:08.573972426 +0000 UTC m=+737.449890014" Jul 09 04:12:17.805220 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:17.805176 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:17.805710 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:17.805244 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:17.817966 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:17.817930 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:12:18.607923 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:12:18.607893 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bm6fh" Jul 09 04:14:51.524446 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:14:51.524366 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:14:51.528123 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:14:51.528090 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:14:51.531404 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:14:51.531376 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:14:51.535009 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:14:51.534990 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:19:51.552153 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:19:51.552121 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:19:51.552934 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:19:51.552912 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:19:51.562206 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:19:51.562181 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:19:51.562882 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:19:51.562832 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:24:51.581263 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:24:51.581230 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:24:51.583765 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:24:51.582151 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:24:51.588033 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:24:51.588011 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:24:51.589090 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:24:51.589067 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:27:05.157557 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.157519 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh"] Jul 09 04:27:05.161104 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.161081 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.163831 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.163808 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d6efa5ba-openshift-default-dockercfg-ztk2d\"" Jul 09 04:27:05.173902 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.173877 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh"] Jul 09 04:27:05.226538 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.226509 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-istio-data\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.226538 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.226539 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/834e905f-c830-4150-a792-810a5992af39-istio-token\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.226720 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.226561 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm88c\" (UniqueName: \"kubernetes.io/projected/834e905f-c830-4150-a792-810a5992af39-kube-api-access-mm88c\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.226720 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.226581 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/834e905f-c830-4150-a792-810a5992af39-istio-podinfo\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.226720 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.226661 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-workload-certs\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.226720 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.226700 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-istio-envoy\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.226875 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.226750 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-credential-socket\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.226875 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.226766 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/834e905f-c830-4150-a792-810a5992af39-istiod-ca-cert\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.226875 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.226790 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-workload-socket\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.328282 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.328246 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-istio-envoy\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.328282 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.328292 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-credential-socket\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.328529 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.328310 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/834e905f-c830-4150-a792-810a5992af39-istiod-ca-cert\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.328529 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.328344 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-workload-socket\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.328529 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.328411 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-istio-data\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.328529 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.328434 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/834e905f-c830-4150-a792-810a5992af39-istio-token\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.328725 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.328575 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm88c\" (UniqueName: \"kubernetes.io/projected/834e905f-c830-4150-a792-810a5992af39-kube-api-access-mm88c\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.328725 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.328624 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/834e905f-c830-4150-a792-810a5992af39-istio-podinfo\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.328725 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.328669 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-workload-certs\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.328919 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.328816 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-credential-socket\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.329006 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.328982 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-workload-socket\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.329068 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.328986 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-istio-data\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.329068 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.329040 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-workload-certs\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.329382 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.329360 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/834e905f-c830-4150-a792-810a5992af39-istiod-ca-cert\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.330543 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.330522 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-istio-envoy\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.331053 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.331037 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/834e905f-c830-4150-a792-810a5992af39-istio-podinfo\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.336817 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.336796 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/834e905f-c830-4150-a792-810a5992af39-istio-token\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.336922 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.336895 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm88c\" (UniqueName: \"kubernetes.io/projected/834e905f-c830-4150-a792-810a5992af39-kube-api-access-mm88c\") pod \"e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.472110 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.472022 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:05.625381 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:27:05.625352 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod834e905f_c830_4150_a792_810a5992af39.slice/crio-89c7370c272de95bdb321249d6aaaa2dc64ae75ddce77767bc4cc2d5dbc7fbed WatchSource:0}: Error finding container 89c7370c272de95bdb321249d6aaaa2dc64ae75ddce77767bc4cc2d5dbc7fbed: Status 404 returned error can't find the container with id 89c7370c272de95bdb321249d6aaaa2dc64ae75ddce77767bc4cc2d5dbc7fbed Jul 09 04:27:05.626025 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.626005 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh"] Jul 09 04:27:05.627819 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.627788 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:27:05.627922 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.627887 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:27:05.627922 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:05.627919 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:27:06.382674 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:06.382636 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" event={"ID":"834e905f-c830-4150-a792-810a5992af39","Type":"ContainerStarted","Data":"82b7e9002374e713a9d88ef3431db1212bef49dcf81defef6e8702aee557558d"} Jul 09 04:27:06.383078 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:06.382698 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" event={"ID":"834e905f-c830-4150-a792-810a5992af39","Type":"ContainerStarted","Data":"89c7370c272de95bdb321249d6aaaa2dc64ae75ddce77767bc4cc2d5dbc7fbed"} Jul 09 04:27:06.407600 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:06.407550 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" podStartSLOduration=1.4075341909999999 podStartE2EDuration="1.407534191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:27:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:27:06.405762203 +0000 UTC m=+1635.281679819" watchObservedRunningTime="2026-07-09 04:27:06.407534191 +0000 UTC m=+1635.283451780" Jul 09 04:27:06.473121 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:06.473084 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:06.479515 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:06.479484 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:07.387201 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:07.387173 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:07.388181 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:07.388162 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:17.459322 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:17.459280 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh"] Jul 09 04:27:17.459793 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:17.459493 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" podUID="834e905f-c830-4150-a792-810a5992af39" containerName="istio-proxy" containerID="cri-o://82b7e9002374e713a9d88ef3431db1212bef49dcf81defef6e8702aee557558d" gracePeriod=30 Jul 09 04:27:18.314156 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.314132 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:18.435994 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.435965 2576 generic.go:356] "Generic (PLEG): container finished" podID="834e905f-c830-4150-a792-810a5992af39" containerID="82b7e9002374e713a9d88ef3431db1212bef49dcf81defef6e8702aee557558d" exitCode=0 Jul 09 04:27:18.436159 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.436009 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" Jul 09 04:27:18.436159 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.436029 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" event={"ID":"834e905f-c830-4150-a792-810a5992af39","Type":"ContainerDied","Data":"82b7e9002374e713a9d88ef3431db1212bef49dcf81defef6e8702aee557558d"} Jul 09 04:27:18.436159 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.436081 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh" event={"ID":"834e905f-c830-4150-a792-810a5992af39","Type":"ContainerDied","Data":"89c7370c272de95bdb321249d6aaaa2dc64ae75ddce77767bc4cc2d5dbc7fbed"} Jul 09 04:27:18.436159 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.436109 2576 scope.go:117] "RemoveContainer" containerID="82b7e9002374e713a9d88ef3431db1212bef49dcf81defef6e8702aee557558d" Jul 09 04:27:18.444355 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.444340 2576 scope.go:117] "RemoveContainer" containerID="82b7e9002374e713a9d88ef3431db1212bef49dcf81defef6e8702aee557558d" Jul 09 04:27:18.444591 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:27:18.444574 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82b7e9002374e713a9d88ef3431db1212bef49dcf81defef6e8702aee557558d\": container with ID starting with 82b7e9002374e713a9d88ef3431db1212bef49dcf81defef6e8702aee557558d not found: ID does not exist" containerID="82b7e9002374e713a9d88ef3431db1212bef49dcf81defef6e8702aee557558d" Jul 09 04:27:18.444645 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.444598 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82b7e9002374e713a9d88ef3431db1212bef49dcf81defef6e8702aee557558d"} err="failed to get container status \"82b7e9002374e713a9d88ef3431db1212bef49dcf81defef6e8702aee557558d\": rpc error: code = NotFound desc = could not find container \"82b7e9002374e713a9d88ef3431db1212bef49dcf81defef6e8702aee557558d\": container with ID starting with 82b7e9002374e713a9d88ef3431db1212bef49dcf81defef6e8702aee557558d not found: ID does not exist" Jul 09 04:27:18.451886 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.451866 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-istio-envoy\") pod \"834e905f-c830-4150-a792-810a5992af39\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " Jul 09 04:27:18.451978 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.451898 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-credential-socket\") pod \"834e905f-c830-4150-a792-810a5992af39\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " Jul 09 04:27:18.451978 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.451965 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-workload-certs\") pod \"834e905f-c830-4150-a792-810a5992af39\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " Jul 09 04:27:18.452083 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.451990 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-workload-socket\") pod \"834e905f-c830-4150-a792-810a5992af39\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " Jul 09 04:27:18.452083 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.452057 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-istio-data\") pod \"834e905f-c830-4150-a792-810a5992af39\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " Jul 09 04:27:18.452180 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.452124 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/834e905f-c830-4150-a792-810a5992af39-istio-token\") pod \"834e905f-c830-4150-a792-810a5992af39\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " Jul 09 04:27:18.452180 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.452164 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/834e905f-c830-4150-a792-810a5992af39-istio-podinfo\") pod \"834e905f-c830-4150-a792-810a5992af39\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " Jul 09 04:27:18.452276 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.452197 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/834e905f-c830-4150-a792-810a5992af39-istiod-ca-cert\") pod \"834e905f-c830-4150-a792-810a5992af39\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " Jul 09 04:27:18.452276 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.452188 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "834e905f-c830-4150-a792-810a5992af39" (UID: "834e905f-c830-4150-a792-810a5992af39"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:27:18.452276 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.452225 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mm88c\" (UniqueName: \"kubernetes.io/projected/834e905f-c830-4150-a792-810a5992af39-kube-api-access-mm88c\") pod \"834e905f-c830-4150-a792-810a5992af39\" (UID: \"834e905f-c830-4150-a792-810a5992af39\") " Jul 09 04:27:18.452276 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.452226 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "834e905f-c830-4150-a792-810a5992af39" (UID: "834e905f-c830-4150-a792-810a5992af39"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:27:18.452276 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.452262 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "834e905f-c830-4150-a792-810a5992af39" (UID: "834e905f-c830-4150-a792-810a5992af39"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:27:18.452586 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.452544 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834e905f-c830-4150-a792-810a5992af39-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "834e905f-c830-4150-a792-810a5992af39" (UID: "834e905f-c830-4150-a792-810a5992af39"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:27:18.452696 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.452590 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-credential-socket\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:27:18.452696 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.452617 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-workload-certs\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:27:18.452696 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.452630 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-workload-socket\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:27:18.452942 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.452909 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-istio-data" (OuterVolumeSpecName: "istio-data") pod "834e905f-c830-4150-a792-810a5992af39" (UID: "834e905f-c830-4150-a792-810a5992af39"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:27:18.454034 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.454011 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "834e905f-c830-4150-a792-810a5992af39" (UID: "834e905f-c830-4150-a792-810a5992af39"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:27:18.454203 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.454187 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/834e905f-c830-4150-a792-810a5992af39-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "834e905f-c830-4150-a792-810a5992af39" (UID: "834e905f-c830-4150-a792-810a5992af39"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:27:18.454249 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.454203 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/834e905f-c830-4150-a792-810a5992af39-kube-api-access-mm88c" (OuterVolumeSpecName: "kube-api-access-mm88c") pod "834e905f-c830-4150-a792-810a5992af39" (UID: "834e905f-c830-4150-a792-810a5992af39"). InnerVolumeSpecName "kube-api-access-mm88c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:27:18.454585 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.454571 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/834e905f-c830-4150-a792-810a5992af39-istio-token" (OuterVolumeSpecName: "istio-token") pod "834e905f-c830-4150-a792-810a5992af39" (UID: "834e905f-c830-4150-a792-810a5992af39"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:27:18.553688 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.553655 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/834e905f-c830-4150-a792-810a5992af39-istio-token\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:27:18.553688 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.553681 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/834e905f-c830-4150-a792-810a5992af39-istio-podinfo\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:27:18.553688 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.553690 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/834e905f-c830-4150-a792-810a5992af39-istiod-ca-cert\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:27:18.553688 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.553698 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm88c\" (UniqueName: \"kubernetes.io/projected/834e905f-c830-4150-a792-810a5992af39-kube-api-access-mm88c\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:27:18.554171 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.553706 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-istio-envoy\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:27:18.554171 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.553714 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/834e905f-c830-4150-a792-810a5992af39-istio-data\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:27:18.758649 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.758612 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh"] Jul 09 04:27:18.761900 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:18.761876 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d6efa5ba-openshift-default-5d9b8b9f75-cbskh"] Jul 09 04:27:19.628857 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:27:19.628753 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="834e905f-c830-4150-a792-810a5992af39" path="/var/lib/kubelet/pods/834e905f-c830-4150-a792-810a5992af39/volumes" Jul 09 04:29:51.607819 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:29:51.607792 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:29:51.610490 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:29:51.610466 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:29:51.616122 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:29:51.616100 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:29:51.618639 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:29:51.618584 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:30:00.133727 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.133686 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs"] Jul 09 04:30:00.134202 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.134053 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="834e905f-c830-4150-a792-810a5992af39" containerName="istio-proxy" Jul 09 04:30:00.134202 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.134065 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="834e905f-c830-4150-a792-810a5992af39" containerName="istio-proxy" Jul 09 04:30:00.134202 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.134117 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="834e905f-c830-4150-a792-810a5992af39" containerName="istio-proxy" Jul 09 04:30:00.137194 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.137169 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg"] Jul 09 04:30:00.137325 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.137292 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs" Jul 09 04:30:00.140283 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.140261 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-svwzx\"" Jul 09 04:30:00.140978 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.140961 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg" Jul 09 04:30:00.144465 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.144439 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs"] Jul 09 04:30:00.157919 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.157884 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg"] Jul 09 04:30:00.214440 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.214396 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b84mr\" (UniqueName: \"kubernetes.io/projected/84e7c345-2993-42e4-acbf-8bfff634b36c-kube-api-access-b84mr\") pod \"maas-api-key-cleanup-29726190-t4stg\" (UID: \"84e7c345-2993-42e4-acbf-8bfff634b36c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg" Jul 09 04:30:00.214624 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.214537 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbxh7\" (UniqueName: \"kubernetes.io/projected/a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5-kube-api-access-mbxh7\") pod \"maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs\" (UID: \"a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs" Jul 09 04:30:00.316002 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.315961 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbxh7\" (UniqueName: \"kubernetes.io/projected/a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5-kube-api-access-mbxh7\") pod \"maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs\" (UID: \"a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs" Jul 09 04:30:00.316176 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.316031 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b84mr\" (UniqueName: \"kubernetes.io/projected/84e7c345-2993-42e4-acbf-8bfff634b36c-kube-api-access-b84mr\") pod \"maas-api-key-cleanup-29726190-t4stg\" (UID: \"84e7c345-2993-42e4-acbf-8bfff634b36c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg" Jul 09 04:30:00.324835 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.324802 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b84mr\" (UniqueName: \"kubernetes.io/projected/84e7c345-2993-42e4-acbf-8bfff634b36c-kube-api-access-b84mr\") pod \"maas-api-key-cleanup-29726190-t4stg\" (UID: \"84e7c345-2993-42e4-acbf-8bfff634b36c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg" Jul 09 04:30:00.324835 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.324857 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbxh7\" (UniqueName: \"kubernetes.io/projected/a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5-kube-api-access-mbxh7\") pod \"maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs\" (UID: \"a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs" Jul 09 04:30:00.450887 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.450829 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs" Jul 09 04:30:00.458851 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.458807 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg" Jul 09 04:30:00.600202 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.600168 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs"] Jul 09 04:30:00.602401 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:30:00.602369 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0b38a4f_d5ca_4e0d_9656_b46dbf1fe2e5.slice/crio-d2ad88e4755d23ce562245383a708e001c4480f8d6f41484e387657bd47c578b WatchSource:0}: Error finding container d2ad88e4755d23ce562245383a708e001c4480f8d6f41484e387657bd47c578b: Status 404 returned error can't find the container with id d2ad88e4755d23ce562245383a708e001c4480f8d6f41484e387657bd47c578b Jul 09 04:30:00.604245 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.604228 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 04:30:00.624386 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:00.624357 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg"] Jul 09 04:30:00.626688 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:30:00.626656 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84e7c345_2993_42e4_acbf_8bfff634b36c.slice/crio-cf09145c4416dd160586a2617679703820b8117d60386ded4459736df10c2a21 WatchSource:0}: Error finding container cf09145c4416dd160586a2617679703820b8117d60386ded4459736df10c2a21: Status 404 returned error can't find the container with id cf09145c4416dd160586a2617679703820b8117d60386ded4459736df10c2a21 Jul 09 04:30:01.121700 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:01.121661 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs" event={"ID":"a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5","Type":"ContainerStarted","Data":"d2ad88e4755d23ce562245383a708e001c4480f8d6f41484e387657bd47c578b"} Jul 09 04:30:01.123046 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:01.123016 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg" event={"ID":"84e7c345-2993-42e4-acbf-8bfff634b36c","Type":"ContainerStarted","Data":"cf09145c4416dd160586a2617679703820b8117d60386ded4459736df10c2a21"} Jul 09 04:30:04.140995 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:04.140961 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg" event={"ID":"84e7c345-2993-42e4-acbf-8bfff634b36c","Type":"ContainerStarted","Data":"f0ec430ce6932c719984eef98be6a792f7cb41764a8b1b39b0c350a2dd9afd82"} Jul 09 04:30:04.142407 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:04.142383 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs" event={"ID":"a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5","Type":"ContainerStarted","Data":"5ace42647613ba4ad6f912bb71b462084e34da7d6de4791cb366f13626992ba9"} Jul 09 04:30:04.156971 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:04.156915 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg" podStartSLOduration=1.735871295 podStartE2EDuration="4.156898302s" totalImagesPullingTime="2.421027007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:30:04.155875607 +0000 UTC m=+1813.031793195" watchObservedRunningTime="2026-07-09 04:30:04.156898302 +0000 UTC m=+1813.032815867" Jul 09 04:30:04.172094 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:04.172043 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs" podStartSLOduration=1.730850136 podStartE2EDuration="4.172024284s" totalImagesPullingTime="2.441174148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:30:04.170480523 +0000 UTC m=+1813.046398112" watchObservedRunningTime="2026-07-09 04:30:04.172024284 +0000 UTC m=+1813.047941894" Jul 09 04:30:32.462198 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:32.462164 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs"] Jul 09 04:30:32.462793 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:30:32.462404 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs" podUID="a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5" containerName="cleanup" containerID="cri-o://5ace42647613ba4ad6f912bb71b462084e34da7d6de4791cb366f13626992ba9" gracePeriod=30 Jul 09 04:31:00.010998 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:00.010958 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg"] Jul 09 04:31:00.011405 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:00.011151 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg" podUID="84e7c345-2993-42e4-acbf-8bfff634b36c" containerName="cleanup" containerID="cri-o://f0ec430ce6932c719984eef98be6a792f7cb41764a8b1b39b0c350a2dd9afd82" gracePeriod=30 Jul 09 04:31:02.647743 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:02.647715 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs" Jul 09 04:31:02.690055 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:02.690020 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbxh7\" (UniqueName: \"kubernetes.io/projected/a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5-kube-api-access-mbxh7\") pod \"a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5\" (UID: \"a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5\") " Jul 09 04:31:02.692342 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:02.692315 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5-kube-api-access-mbxh7" (OuterVolumeSpecName: "kube-api-access-mbxh7") pod "a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5" (UID: "a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5"). InnerVolumeSpecName "kube-api-access-mbxh7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:31:02.790746 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:02.790714 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbxh7\" (UniqueName: \"kubernetes.io/projected/a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5-kube-api-access-mbxh7\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:31:03.413178 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:03.413139 2576 generic.go:356] "Generic (PLEG): container finished" podID="a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5" containerID="5ace42647613ba4ad6f912bb71b462084e34da7d6de4791cb366f13626992ba9" exitCode=137 Jul 09 04:31:03.413424 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:03.413185 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs" Jul 09 04:31:03.413424 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:03.413224 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs" event={"ID":"a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5","Type":"ContainerDied","Data":"5ace42647613ba4ad6f912bb71b462084e34da7d6de4791cb366f13626992ba9"} Jul 09 04:31:03.413424 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:03.413290 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs" event={"ID":"a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5","Type":"ContainerDied","Data":"d2ad88e4755d23ce562245383a708e001c4480f8d6f41484e387657bd47c578b"} Jul 09 04:31:03.413424 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:03.413313 2576 scope.go:117] "RemoveContainer" containerID="5ace42647613ba4ad6f912bb71b462084e34da7d6de4791cb366f13626992ba9" Jul 09 04:31:03.421559 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:03.421539 2576 scope.go:117] "RemoveContainer" containerID="5ace42647613ba4ad6f912bb71b462084e34da7d6de4791cb366f13626992ba9" Jul 09 04:31:03.421809 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:31:03.421792 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ace42647613ba4ad6f912bb71b462084e34da7d6de4791cb366f13626992ba9\": container with ID starting with 5ace42647613ba4ad6f912bb71b462084e34da7d6de4791cb366f13626992ba9 not found: ID does not exist" containerID="5ace42647613ba4ad6f912bb71b462084e34da7d6de4791cb366f13626992ba9" Jul 09 04:31:03.421902 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:03.421816 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ace42647613ba4ad6f912bb71b462084e34da7d6de4791cb366f13626992ba9"} err="failed to get container status \"5ace42647613ba4ad6f912bb71b462084e34da7d6de4791cb366f13626992ba9\": rpc error: code = NotFound desc = could not find container \"5ace42647613ba4ad6f912bb71b462084e34da7d6de4791cb366f13626992ba9\": container with ID starting with 5ace42647613ba4ad6f912bb71b462084e34da7d6de4791cb366f13626992ba9 not found: ID does not exist" Jul 09 04:31:03.433246 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:03.433223 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs"] Jul 09 04:31:03.436123 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:03.436101 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-e747c320-29726190-ttkgs"] Jul 09 04:31:03.623873 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:03.623824 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5" path="/var/lib/kubelet/pods/a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5/volumes" Jul 09 04:31:30.197580 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:30.197553 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg" Jul 09 04:31:30.341169 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:30.341072 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b84mr\" (UniqueName: \"kubernetes.io/projected/84e7c345-2993-42e4-acbf-8bfff634b36c-kube-api-access-b84mr\") pod \"84e7c345-2993-42e4-acbf-8bfff634b36c\" (UID: \"84e7c345-2993-42e4-acbf-8bfff634b36c\") " Jul 09 04:31:30.343249 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:30.343222 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84e7c345-2993-42e4-acbf-8bfff634b36c-kube-api-access-b84mr" (OuterVolumeSpecName: "kube-api-access-b84mr") pod "84e7c345-2993-42e4-acbf-8bfff634b36c" (UID: "84e7c345-2993-42e4-acbf-8bfff634b36c"). InnerVolumeSpecName "kube-api-access-b84mr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:31:30.442723 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:30.442688 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b84mr\" (UniqueName: \"kubernetes.io/projected/84e7c345-2993-42e4-acbf-8bfff634b36c-kube-api-access-b84mr\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:31:30.531866 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:30.531819 2576 generic.go:356] "Generic (PLEG): container finished" podID="84e7c345-2993-42e4-acbf-8bfff634b36c" containerID="f0ec430ce6932c719984eef98be6a792f7cb41764a8b1b39b0c350a2dd9afd82" exitCode=137 Jul 09 04:31:30.532036 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:30.531877 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg" Jul 09 04:31:30.532036 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:30.531911 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg" event={"ID":"84e7c345-2993-42e4-acbf-8bfff634b36c","Type":"ContainerDied","Data":"f0ec430ce6932c719984eef98be6a792f7cb41764a8b1b39b0c350a2dd9afd82"} Jul 09 04:31:30.532036 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:30.531967 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg" event={"ID":"84e7c345-2993-42e4-acbf-8bfff634b36c","Type":"ContainerDied","Data":"cf09145c4416dd160586a2617679703820b8117d60386ded4459736df10c2a21"} Jul 09 04:31:30.532036 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:30.531995 2576 scope.go:117] "RemoveContainer" containerID="f0ec430ce6932c719984eef98be6a792f7cb41764a8b1b39b0c350a2dd9afd82" Jul 09 04:31:30.540360 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:30.540342 2576 scope.go:117] "RemoveContainer" containerID="f0ec430ce6932c719984eef98be6a792f7cb41764a8b1b39b0c350a2dd9afd82" Jul 09 04:31:30.540646 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:31:30.540624 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0ec430ce6932c719984eef98be6a792f7cb41764a8b1b39b0c350a2dd9afd82\": container with ID starting with f0ec430ce6932c719984eef98be6a792f7cb41764a8b1b39b0c350a2dd9afd82 not found: ID does not exist" containerID="f0ec430ce6932c719984eef98be6a792f7cb41764a8b1b39b0c350a2dd9afd82" Jul 09 04:31:30.540685 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:30.540656 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0ec430ce6932c719984eef98be6a792f7cb41764a8b1b39b0c350a2dd9afd82"} err="failed to get container status \"f0ec430ce6932c719984eef98be6a792f7cb41764a8b1b39b0c350a2dd9afd82\": rpc error: code = NotFound desc = could not find container \"f0ec430ce6932c719984eef98be6a792f7cb41764a8b1b39b0c350a2dd9afd82\": container with ID starting with f0ec430ce6932c719984eef98be6a792f7cb41764a8b1b39b0c350a2dd9afd82 not found: ID does not exist" Jul 09 04:31:30.552209 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:30.552180 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg"] Jul 09 04:31:30.555968 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:30.555943 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29726190-t4stg"] Jul 09 04:31:31.624682 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:31:31.624644 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84e7c345-2993-42e4-acbf-8bfff634b36c" path="/var/lib/kubelet/pods/84e7c345-2993-42e4-acbf-8bfff634b36c/volumes" Jul 09 04:32:19.374269 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.374232 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5"] Jul 09 04:32:19.376624 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.374551 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84e7c345-2993-42e4-acbf-8bfff634b36c" containerName="cleanup" Jul 09 04:32:19.376624 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.374562 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="84e7c345-2993-42e4-acbf-8bfff634b36c" containerName="cleanup" Jul 09 04:32:19.376624 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.374576 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5" containerName="cleanup" Jul 09 04:32:19.376624 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.374581 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5" containerName="cleanup" Jul 09 04:32:19.376624 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.374663 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="84e7c345-2993-42e4-acbf-8bfff634b36c" containerName="cleanup" Jul 09 04:32:19.376624 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.374670 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0b38a4f-d5ca-4e0d-9656-b46dbf1fe2e5" containerName="cleanup" Jul 09 04:32:19.377559 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.377541 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.380565 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.380543 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c184fc-openshift-default-dockercfg-zqnnz\"" Jul 09 04:32:19.390461 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.390434 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5"] Jul 09 04:32:19.473461 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.473424 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-workload-socket\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.473461 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.473463 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-workload-certs\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.473703 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.473514 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-token\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.473703 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.473544 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbx7f\" (UniqueName: \"kubernetes.io/projected/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-kube-api-access-rbx7f\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.473703 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.473601 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-credential-socket\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.473703 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.473642 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istiod-ca-cert\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.473703 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.473675 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-podinfo\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.473990 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.473718 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-data\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.473990 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.473754 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-envoy\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.574267 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.574223 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-token\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.574267 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.574273 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbx7f\" (UniqueName: \"kubernetes.io/projected/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-kube-api-access-rbx7f\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.574536 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.574317 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-credential-socket\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.574536 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.574342 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istiod-ca-cert\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.574536 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.574367 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-podinfo\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.574536 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.574411 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-data\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.574536 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.574457 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-envoy\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.574536 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.574497 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-workload-socket\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.574871 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.574695 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-workload-certs\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.574871 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.574795 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-credential-socket\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.574976 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.574960 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-data\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.575031 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.575011 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-workload-socket\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.575181 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.575160 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istiod-ca-cert\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.575251 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.575236 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-workload-certs\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.576937 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.576912 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-envoy\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.577114 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.577095 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-podinfo\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.582565 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.582543 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-token\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.582662 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.582566 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbx7f\" (UniqueName: \"kubernetes.io/projected/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-kube-api-access-rbx7f\") pod \"e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.689191 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.689158 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:19.829934 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.829906 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5"] Jul 09 04:32:19.831862 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:32:19.831813 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19b58b1f_3192_4f78_ad5e_de04bc9dfb97.slice/crio-f6f09ad7739243779e7977560956a96d513de51c3d875bf409eb4977c896ec45 WatchSource:0}: Error finding container f6f09ad7739243779e7977560956a96d513de51c3d875bf409eb4977c896ec45: Status 404 returned error can't find the container with id f6f09ad7739243779e7977560956a96d513de51c3d875bf409eb4977c896ec45 Jul 09 04:32:19.833921 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.833885 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:32:19.834013 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.833950 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:32:19.834013 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:19.833978 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:32:20.753386 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:20.753343 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" event={"ID":"19b58b1f-3192-4f78-ad5e-de04bc9dfb97","Type":"ContainerStarted","Data":"5c432b58b051c6cb8ded26201c9db20dc4a924deeba8c2165d93bd6c07afca0c"} Jul 09 04:32:20.753777 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:20.753435 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" event={"ID":"19b58b1f-3192-4f78-ad5e-de04bc9dfb97","Type":"ContainerStarted","Data":"f6f09ad7739243779e7977560956a96d513de51c3d875bf409eb4977c896ec45"} Jul 09 04:32:20.774303 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:20.774250 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" podStartSLOduration=1.774227988 podStartE2EDuration="1.774227988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:32:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:32:20.771785677 +0000 UTC m=+1949.647703268" watchObservedRunningTime="2026-07-09 04:32:20.774227988 +0000 UTC m=+1949.650145576" Jul 09 04:32:21.690252 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:21.690210 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:21.695486 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:21.695459 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:21.757615 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:21.757574 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:21.758712 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:21.758687 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:37.799020 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:37.798980 2576 patch_prober.go:28] interesting pod/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 04:32:37.799429 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:37.799076 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" podUID="19b58b1f-3192-4f78-ad5e-de04bc9dfb97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 04:32:38.837125 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:38.837093 2576 generic.go:356] "Generic (PLEG): container finished" podID="19b58b1f-3192-4f78-ad5e-de04bc9dfb97" containerID="5c432b58b051c6cb8ded26201c9db20dc4a924deeba8c2165d93bd6c07afca0c" exitCode=0 Jul 09 04:32:38.837644 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:38.837145 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" event={"ID":"19b58b1f-3192-4f78-ad5e-de04bc9dfb97","Type":"ContainerDied","Data":"5c432b58b051c6cb8ded26201c9db20dc4a924deeba8c2165d93bd6c07afca0c"} Jul 09 04:32:38.837644 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:38.837624 2576 scope.go:117] "RemoveContainer" containerID="5c432b58b051c6cb8ded26201c9db20dc4a924deeba8c2165d93bd6c07afca0c" Jul 09 04:32:38.838654 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:38.838627 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:32:38.838755 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:38.838697 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:32:38.838755 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:38.838728 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:32:39.845728 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:39.845681 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" event={"ID":"19b58b1f-3192-4f78-ad5e-de04bc9dfb97","Type":"ContainerStarted","Data":"696cd29c8dd24ef959d06bf933f7b86ab3e97063762cdf60b18689f336f41735"} Jul 09 04:32:40.690224 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:40.690181 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:40.690427 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:40.690408 2576 patch_prober.go:28] interesting pod/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 09 04:32:40.690483 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:40.690454 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" podUID="19b58b1f-3192-4f78-ad5e-de04bc9dfb97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 09 04:32:41.689586 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:41.689546 2576 patch_prober.go:28] interesting pod/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 09 04:32:41.690014 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:41.689603 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" podUID="19b58b1f-3192-4f78-ad5e-de04bc9dfb97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 09 04:32:42.689741 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:42.689701 2576 patch_prober.go:28] interesting pod/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body= Jul 09 04:32:42.690233 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:42.689762 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" podUID="19b58b1f-3192-4f78-ad5e-de04bc9dfb97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" Jul 09 04:32:43.699784 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:43.699743 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:43.700263 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:43.700199 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:32:43.700948 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:32:43.700929 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:34:06.630523 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:06.630479 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5"] Jul 09 04:34:06.631031 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:06.630740 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" podUID="19b58b1f-3192-4f78-ad5e-de04bc9dfb97" containerName="istio-proxy" containerID="cri-o://696cd29c8dd24ef959d06bf933f7b86ab3e97063762cdf60b18689f336f41735" gracePeriod=30 Jul 09 04:34:11.883333 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.883297 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:34:11.977557 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.977458 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-podinfo\") pod \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " Jul 09 04:34:11.977557 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.977527 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-workload-socket\") pod \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " Jul 09 04:34:11.977557 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.977546 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-workload-certs\") pod \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " Jul 09 04:34:11.977878 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.977565 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-envoy\") pod \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " Jul 09 04:34:11.977878 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.977685 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbx7f\" (UniqueName: \"kubernetes.io/projected/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-kube-api-access-rbx7f\") pod \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " Jul 09 04:34:11.977878 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.977722 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-token\") pod \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " Jul 09 04:34:11.977878 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.977779 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-credential-socket\") pod \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " Jul 09 04:34:11.977878 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.977822 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istiod-ca-cert\") pod \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " Jul 09 04:34:11.977878 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.977875 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-data\") pod \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\" (UID: \"19b58b1f-3192-4f78-ad5e-de04bc9dfb97\") " Jul 09 04:34:11.978186 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.977940 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "19b58b1f-3192-4f78-ad5e-de04bc9dfb97" (UID: "19b58b1f-3192-4f78-ad5e-de04bc9dfb97"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:34:11.978186 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.977979 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "19b58b1f-3192-4f78-ad5e-de04bc9dfb97" (UID: "19b58b1f-3192-4f78-ad5e-de04bc9dfb97"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:34:11.978295 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.978260 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-workload-socket\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:11.978295 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.978270 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "19b58b1f-3192-4f78-ad5e-de04bc9dfb97" (UID: "19b58b1f-3192-4f78-ad5e-de04bc9dfb97"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:34:11.978295 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.978280 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-workload-certs\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:11.978437 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.978389 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "19b58b1f-3192-4f78-ad5e-de04bc9dfb97" (UID: "19b58b1f-3192-4f78-ad5e-de04bc9dfb97"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:34:11.978858 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.978806 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-data" (OuterVolumeSpecName: "istio-data") pod "19b58b1f-3192-4f78-ad5e-de04bc9dfb97" (UID: "19b58b1f-3192-4f78-ad5e-de04bc9dfb97"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:34:11.979902 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.979870 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "19b58b1f-3192-4f78-ad5e-de04bc9dfb97" (UID: "19b58b1f-3192-4f78-ad5e-de04bc9dfb97"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:34:11.979902 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.979883 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "19b58b1f-3192-4f78-ad5e-de04bc9dfb97" (UID: "19b58b1f-3192-4f78-ad5e-de04bc9dfb97"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:34:11.980316 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.980294 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-token" (OuterVolumeSpecName: "istio-token") pod "19b58b1f-3192-4f78-ad5e-de04bc9dfb97" (UID: "19b58b1f-3192-4f78-ad5e-de04bc9dfb97"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:34:11.980374 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:11.980306 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-kube-api-access-rbx7f" (OuterVolumeSpecName: "kube-api-access-rbx7f") pod "19b58b1f-3192-4f78-ad5e-de04bc9dfb97" (UID: "19b58b1f-3192-4f78-ad5e-de04bc9dfb97"). InnerVolumeSpecName "kube-api-access-rbx7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:34:12.079110 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.079078 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-podinfo\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:12.079110 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.079103 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-envoy\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:12.079110 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.079111 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbx7f\" (UniqueName: \"kubernetes.io/projected/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-kube-api-access-rbx7f\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:12.079110 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.079120 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-token\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:12.079385 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.079129 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-credential-socket\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:12.079385 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.079137 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istiod-ca-cert\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:12.079385 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.079145 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19b58b1f-3192-4f78-ad5e-de04bc9dfb97-istio-data\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:12.250596 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.250512 2576 generic.go:356] "Generic (PLEG): container finished" podID="19b58b1f-3192-4f78-ad5e-de04bc9dfb97" containerID="696cd29c8dd24ef959d06bf933f7b86ab3e97063762cdf60b18689f336f41735" exitCode=0 Jul 09 04:34:12.250596 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.250560 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" Jul 09 04:34:12.250596 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.250578 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" event={"ID":"19b58b1f-3192-4f78-ad5e-de04bc9dfb97","Type":"ContainerDied","Data":"696cd29c8dd24ef959d06bf933f7b86ab3e97063762cdf60b18689f336f41735"} Jul 09 04:34:12.250919 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.250621 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5" event={"ID":"19b58b1f-3192-4f78-ad5e-de04bc9dfb97","Type":"ContainerDied","Data":"f6f09ad7739243779e7977560956a96d513de51c3d875bf409eb4977c896ec45"} Jul 09 04:34:12.250919 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.250642 2576 scope.go:117] "RemoveContainer" containerID="696cd29c8dd24ef959d06bf933f7b86ab3e97063762cdf60b18689f336f41735" Jul 09 04:34:12.259727 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.259707 2576 scope.go:117] "RemoveContainer" containerID="5c432b58b051c6cb8ded26201c9db20dc4a924deeba8c2165d93bd6c07afca0c" Jul 09 04:34:12.267683 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.267665 2576 scope.go:117] "RemoveContainer" containerID="696cd29c8dd24ef959d06bf933f7b86ab3e97063762cdf60b18689f336f41735" Jul 09 04:34:12.267963 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:34:12.267942 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"696cd29c8dd24ef959d06bf933f7b86ab3e97063762cdf60b18689f336f41735\": container with ID starting with 696cd29c8dd24ef959d06bf933f7b86ab3e97063762cdf60b18689f336f41735 not found: ID does not exist" containerID="696cd29c8dd24ef959d06bf933f7b86ab3e97063762cdf60b18689f336f41735" Jul 09 04:34:12.268043 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.267972 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"696cd29c8dd24ef959d06bf933f7b86ab3e97063762cdf60b18689f336f41735"} err="failed to get container status \"696cd29c8dd24ef959d06bf933f7b86ab3e97063762cdf60b18689f336f41735\": rpc error: code = NotFound desc = could not find container \"696cd29c8dd24ef959d06bf933f7b86ab3e97063762cdf60b18689f336f41735\": container with ID starting with 696cd29c8dd24ef959d06bf933f7b86ab3e97063762cdf60b18689f336f41735 not found: ID does not exist" Jul 09 04:34:12.268043 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.267992 2576 scope.go:117] "RemoveContainer" containerID="5c432b58b051c6cb8ded26201c9db20dc4a924deeba8c2165d93bd6c07afca0c" Jul 09 04:34:12.268243 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:34:12.268224 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c432b58b051c6cb8ded26201c9db20dc4a924deeba8c2165d93bd6c07afca0c\": container with ID starting with 5c432b58b051c6cb8ded26201c9db20dc4a924deeba8c2165d93bd6c07afca0c not found: ID does not exist" containerID="5c432b58b051c6cb8ded26201c9db20dc4a924deeba8c2165d93bd6c07afca0c" Jul 09 04:34:12.268283 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.268249 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c432b58b051c6cb8ded26201c9db20dc4a924deeba8c2165d93bd6c07afca0c"} err="failed to get container status \"5c432b58b051c6cb8ded26201c9db20dc4a924deeba8c2165d93bd6c07afca0c\": rpc error: code = NotFound desc = could not find container \"5c432b58b051c6cb8ded26201c9db20dc4a924deeba8c2165d93bd6c07afca0c\": container with ID starting with 5c432b58b051c6cb8ded26201c9db20dc4a924deeba8c2165d93bd6c07afca0c not found: ID does not exist" Jul 09 04:34:12.273036 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.273014 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5"] Jul 09 04:34:12.276727 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:12.276705 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c184fc-openshift-default-6985fc9b9f-szbh5"] Jul 09 04:34:13.623988 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:13.623912 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19b58b1f-3192-4f78-ad5e-de04bc9dfb97" path="/var/lib/kubelet/pods/19b58b1f-3192-4f78-ad5e-de04bc9dfb97/volumes" Jul 09 04:34:14.383016 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.382983 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv"] Jul 09 04:34:14.383315 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.383300 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19b58b1f-3192-4f78-ad5e-de04bc9dfb97" containerName="istio-proxy" Jul 09 04:34:14.383376 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.383317 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="19b58b1f-3192-4f78-ad5e-de04bc9dfb97" containerName="istio-proxy" Jul 09 04:34:14.383376 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.383331 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19b58b1f-3192-4f78-ad5e-de04bc9dfb97" containerName="istio-proxy" Jul 09 04:34:14.383376 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.383336 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="19b58b1f-3192-4f78-ad5e-de04bc9dfb97" containerName="istio-proxy" Jul 09 04:34:14.383522 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.383401 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="19b58b1f-3192-4f78-ad5e-de04bc9dfb97" containerName="istio-proxy" Jul 09 04:34:14.387801 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.387781 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.390499 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.390479 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-dac2f3-openshift-default-dockercfg-l47gz\"" Jul 09 04:34:14.397827 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.397804 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv"] Jul 09 04:34:14.499532 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.499499 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istiod-ca-cert\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.499532 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.499533 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cvn7\" (UniqueName: \"kubernetes.io/projected/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-kube-api-access-7cvn7\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.499785 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.499556 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-data\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.499785 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.499576 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-token\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.499785 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.499651 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-workload-certs\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.499785 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.499714 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-envoy\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.499785 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.499741 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-workload-socket\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.500041 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.499802 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-credential-socket\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.500041 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.499853 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-podinfo\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.601091 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.601049 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-envoy\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.601091 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.601092 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-workload-socket\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.601352 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.601145 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-credential-socket\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.601352 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.601172 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-podinfo\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.601352 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.601222 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istiod-ca-cert\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.601352 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.601341 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cvn7\" (UniqueName: \"kubernetes.io/projected/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-kube-api-access-7cvn7\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.601569 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.601395 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-data\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.601569 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.601427 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-token\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.601569 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.601461 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-workload-certs\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.601569 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.601552 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-credential-socket\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.601753 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.601592 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-workload-socket\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.601753 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.601737 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-data\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.602135 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.602109 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istiod-ca-cert\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.602225 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.602195 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-workload-certs\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.603491 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.603467 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-envoy\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.603716 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.603699 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-podinfo\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.626286 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.626254 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cvn7\" (UniqueName: \"kubernetes.io/projected/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-kube-api-access-7cvn7\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.626286 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.626283 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-token\") pod \"e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.699068 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.699033 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:14.833102 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.833070 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv"] Jul 09 04:34:14.835679 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:34:14.835652 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3b1efcc_1f76_47fb_ab4e_fb8895ab339a.slice/crio-6df20c0538dab6f909f86d12507b53d32c88ba68d171ddf3e995971427171046 WatchSource:0}: Error finding container 6df20c0538dab6f909f86d12507b53d32c88ba68d171ddf3e995971427171046: Status 404 returned error can't find the container with id 6df20c0538dab6f909f86d12507b53d32c88ba68d171ddf3e995971427171046 Jul 09 04:34:14.837746 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.837718 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:34:14.837796 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.837779 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:34:14.837847 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:14.837812 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:34:15.268361 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:15.268324 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" event={"ID":"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a","Type":"ContainerStarted","Data":"7fe07b08073861fcb7e6a6a90bdf5403b763a2e188eef4a40dec60f92d13f4d2"} Jul 09 04:34:15.268550 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:15.268412 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" event={"ID":"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a","Type":"ContainerStarted","Data":"6df20c0538dab6f909f86d12507b53d32c88ba68d171ddf3e995971427171046"} Jul 09 04:34:15.289896 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:15.289832 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" podStartSLOduration=1.2898147500000001 podStartE2EDuration="1.28981475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:34:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:34:15.287415967 +0000 UTC m=+2064.163333569" watchObservedRunningTime="2026-07-09 04:34:15.28981475 +0000 UTC m=+2064.165732337" Jul 09 04:34:15.700041 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:15.700005 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:15.705163 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:15.705132 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:16.272318 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:16.272287 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:16.273481 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:16.273459 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:32.314047 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:32.314002 2576 patch_prober.go:28] interesting pod/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 04:34:32.314429 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:32.314077 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" podUID="a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 04:34:34.367344 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:34.367304 2576 generic.go:356] "Generic (PLEG): container finished" podID="a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" containerID="7fe07b08073861fcb7e6a6a90bdf5403b763a2e188eef4a40dec60f92d13f4d2" exitCode=0 Jul 09 04:34:34.367763 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:34.367384 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" event={"ID":"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a","Type":"ContainerDied","Data":"7fe07b08073861fcb7e6a6a90bdf5403b763a2e188eef4a40dec60f92d13f4d2"} Jul 09 04:34:34.367806 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:34.367791 2576 scope.go:117] "RemoveContainer" containerID="7fe07b08073861fcb7e6a6a90bdf5403b763a2e188eef4a40dec60f92d13f4d2" Jul 09 04:34:34.368921 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:34.368892 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:34:34.369030 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:34.368962 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:34:34.369030 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:34.368994 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:34:35.374243 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:35.374207 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" event={"ID":"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a","Type":"ContainerStarted","Data":"dab1430e515e7273958dc89225abae5496fd5c918e6a5080599a1e46040309d1"} Jul 09 04:34:35.699627 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:35.699587 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:35.699890 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:35.699832 2576 patch_prober.go:28] interesting pod/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 09 04:34:35.699967 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:35.699920 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" podUID="a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 09 04:34:36.699864 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:36.699803 2576 patch_prober.go:28] interesting pod/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 09 04:34:36.700293 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:36.699899 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" podUID="a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 09 04:34:37.700193 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:37.700155 2576 patch_prober.go:28] interesting pod/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 09 04:34:37.700592 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:37.700212 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" podUID="a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 09 04:34:38.699467 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:38.699420 2576 patch_prober.go:28] interesting pod/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 09 04:34:38.699645 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:38.699497 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" podUID="a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 09 04:34:39.703763 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:39.703721 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:39.704272 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:39.704068 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:39.705045 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:39.705015 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:44.343196 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:44.343160 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv"] Jul 09 04:34:44.343785 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:44.343357 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" podUID="a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" containerName="istio-proxy" containerID="cri-o://dab1430e515e7273958dc89225abae5496fd5c918e6a5080599a1e46040309d1" gracePeriod=30 Jul 09 04:34:49.596053 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.596024 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:49.636245 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.636164 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-envoy\") pod \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " Jul 09 04:34:49.636245 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.636198 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-workload-socket\") pod \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " Jul 09 04:34:49.636245 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.636223 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-credential-socket\") pod \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " Jul 09 04:34:49.636498 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.636249 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-workload-certs\") pod \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " Jul 09 04:34:49.636498 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.636276 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-podinfo\") pod \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " Jul 09 04:34:49.636498 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.636305 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-token\") pod \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " Jul 09 04:34:49.636498 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.636348 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istiod-ca-cert\") pod \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " Jul 09 04:34:49.636498 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.636376 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-data\") pod \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " Jul 09 04:34:49.636498 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.636406 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cvn7\" (UniqueName: \"kubernetes.io/projected/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-kube-api-access-7cvn7\") pod \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\" (UID: \"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a\") " Jul 09 04:34:49.636792 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.636491 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" (UID: "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:34:49.636792 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.636699 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-workload-socket\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:49.636792 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.636747 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" (UID: "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:34:49.638948 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.637332 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" (UID: "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:34:49.638948 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.637336 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" (UID: "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:34:49.638948 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.638098 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-data" (OuterVolumeSpecName: "istio-data") pod "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" (UID: "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:34:49.641898 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.639600 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" (UID: "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:34:49.641898 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.639826 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-token" (OuterVolumeSpecName: "istio-token") pod "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" (UID: "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:34:49.641898 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.640036 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-kube-api-access-7cvn7" (OuterVolumeSpecName: "kube-api-access-7cvn7") pod "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" (UID: "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a"). InnerVolumeSpecName "kube-api-access-7cvn7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:34:49.641898 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.640038 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" (UID: "a3b1efcc-1f76-47fb-ab4e-fb8895ab339a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:34:49.737228 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.737191 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-podinfo\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:49.737228 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.737226 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-token\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:49.737228 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.737234 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istiod-ca-cert\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:49.737228 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.737245 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-data\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:49.737510 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.737259 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cvn7\" (UniqueName: \"kubernetes.io/projected/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-kube-api-access-7cvn7\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:49.737510 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.737270 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-istio-envoy\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:49.737510 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.737279 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-credential-socket\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:49.737510 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:49.737288 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a-workload-certs\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:34:50.444330 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:50.444289 2576 generic.go:356] "Generic (PLEG): container finished" podID="a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" containerID="dab1430e515e7273958dc89225abae5496fd5c918e6a5080599a1e46040309d1" exitCode=0 Jul 09 04:34:50.444537 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:50.444340 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" event={"ID":"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a","Type":"ContainerDied","Data":"dab1430e515e7273958dc89225abae5496fd5c918e6a5080599a1e46040309d1"} Jul 09 04:34:50.444537 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:50.444362 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" Jul 09 04:34:50.444537 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:50.444413 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv" event={"ID":"a3b1efcc-1f76-47fb-ab4e-fb8895ab339a","Type":"ContainerDied","Data":"6df20c0538dab6f909f86d12507b53d32c88ba68d171ddf3e995971427171046"} Jul 09 04:34:50.444537 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:50.444437 2576 scope.go:117] "RemoveContainer" containerID="dab1430e515e7273958dc89225abae5496fd5c918e6a5080599a1e46040309d1" Jul 09 04:34:50.453497 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:50.453477 2576 scope.go:117] "RemoveContainer" containerID="7fe07b08073861fcb7e6a6a90bdf5403b763a2e188eef4a40dec60f92d13f4d2" Jul 09 04:34:50.461302 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:50.461283 2576 scope.go:117] "RemoveContainer" containerID="dab1430e515e7273958dc89225abae5496fd5c918e6a5080599a1e46040309d1" Jul 09 04:34:50.461573 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:34:50.461553 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dab1430e515e7273958dc89225abae5496fd5c918e6a5080599a1e46040309d1\": container with ID starting with dab1430e515e7273958dc89225abae5496fd5c918e6a5080599a1e46040309d1 not found: ID does not exist" containerID="dab1430e515e7273958dc89225abae5496fd5c918e6a5080599a1e46040309d1" Jul 09 04:34:50.461609 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:50.461583 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dab1430e515e7273958dc89225abae5496fd5c918e6a5080599a1e46040309d1"} err="failed to get container status \"dab1430e515e7273958dc89225abae5496fd5c918e6a5080599a1e46040309d1\": rpc error: code = NotFound desc = could not find container \"dab1430e515e7273958dc89225abae5496fd5c918e6a5080599a1e46040309d1\": container with ID starting with dab1430e515e7273958dc89225abae5496fd5c918e6a5080599a1e46040309d1 not found: ID does not exist" Jul 09 04:34:50.461609 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:50.461602 2576 scope.go:117] "RemoveContainer" containerID="7fe07b08073861fcb7e6a6a90bdf5403b763a2e188eef4a40dec60f92d13f4d2" Jul 09 04:34:50.461849 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:34:50.461819 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fe07b08073861fcb7e6a6a90bdf5403b763a2e188eef4a40dec60f92d13f4d2\": container with ID starting with 7fe07b08073861fcb7e6a6a90bdf5403b763a2e188eef4a40dec60f92d13f4d2 not found: ID does not exist" containerID="7fe07b08073861fcb7e6a6a90bdf5403b763a2e188eef4a40dec60f92d13f4d2" Jul 09 04:34:50.461893 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:50.461859 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fe07b08073861fcb7e6a6a90bdf5403b763a2e188eef4a40dec60f92d13f4d2"} err="failed to get container status \"7fe07b08073861fcb7e6a6a90bdf5403b763a2e188eef4a40dec60f92d13f4d2\": rpc error: code = NotFound desc = could not find container \"7fe07b08073861fcb7e6a6a90bdf5403b763a2e188eef4a40dec60f92d13f4d2\": container with ID starting with 7fe07b08073861fcb7e6a6a90bdf5403b763a2e188eef4a40dec60f92d13f4d2 not found: ID does not exist" Jul 09 04:34:50.466333 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:50.466309 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv"] Jul 09 04:34:50.470221 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:50.470199 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-dac2f3-openshift-default-7b85875585-4n5gv"] Jul 09 04:34:51.624468 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:51.624438 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" path="/var/lib/kubelet/pods/a3b1efcc-1f76-47fb-ab4e-fb8895ab339a/volumes" Jul 09 04:34:51.638117 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:51.638093 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:34:51.640764 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:51.640743 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:34:51.645088 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:51.645062 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:34:51.648274 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:51.648251 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:34:52.307806 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.307769 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv"] Jul 09 04:34:52.308174 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.308159 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" containerName="istio-proxy" Jul 09 04:34:52.308255 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.308176 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" containerName="istio-proxy" Jul 09 04:34:52.308255 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.308189 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" containerName="istio-proxy" Jul 09 04:34:52.308255 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.308195 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" containerName="istio-proxy" Jul 09 04:34:52.308255 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.308245 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="19b58b1f-3192-4f78-ad5e-de04bc9dfb97" containerName="istio-proxy" Jul 09 04:34:52.308255 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.308255 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" containerName="istio-proxy" Jul 09 04:34:52.308479 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.308266 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3b1efcc-1f76-47fb-ab4e-fb8895ab339a" containerName="istio-proxy" Jul 09 04:34:52.312974 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.312950 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.315588 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.315571 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-7b94ae-openshift-default-dockercfg-vshv2\"" Jul 09 04:34:52.322802 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.322777 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv"] Jul 09 04:34:52.360389 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.360351 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-workload-certs\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.360389 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.360387 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmxrj\" (UniqueName: \"kubernetes.io/projected/e86caf21-974c-444c-8b30-f553c77cd1be-kube-api-access-wmxrj\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.360665 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.360427 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-istio-data\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.360665 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.360452 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-credential-socket\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.360665 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.360477 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e86caf21-974c-444c-8b30-f553c77cd1be-istio-token\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.360665 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.360553 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-istio-envoy\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.360665 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.360582 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-workload-socket\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.360665 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.360603 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e86caf21-974c-444c-8b30-f553c77cd1be-istio-podinfo\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.360665 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.360622 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e86caf21-974c-444c-8b30-f553c77cd1be-istiod-ca-cert\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.462062 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.462032 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e86caf21-974c-444c-8b30-f553c77cd1be-istio-token\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.462295 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.462081 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-istio-envoy\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.462295 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.462202 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-workload-socket\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.462295 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.462238 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e86caf21-974c-444c-8b30-f553c77cd1be-istio-podinfo\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.462295 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.462254 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e86caf21-974c-444c-8b30-f553c77cd1be-istiod-ca-cert\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.462507 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.462318 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-workload-certs\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.462507 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.462337 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmxrj\" (UniqueName: \"kubernetes.io/projected/e86caf21-974c-444c-8b30-f553c77cd1be-kube-api-access-wmxrj\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.462507 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.462377 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-istio-data\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.462507 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.462406 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-credential-socket\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.462711 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.462595 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-workload-socket\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.462711 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.462692 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-workload-certs\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.462801 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.462727 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-credential-socket\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.462943 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.462921 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-istio-data\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.463093 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.463077 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e86caf21-974c-444c-8b30-f553c77cd1be-istiod-ca-cert\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.464346 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.464318 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-istio-envoy\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.464593 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.464577 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e86caf21-974c-444c-8b30-f553c77cd1be-istio-podinfo\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.470915 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.470881 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e86caf21-974c-444c-8b30-f553c77cd1be-istio-token\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.471037 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.470947 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmxrj\" (UniqueName: \"kubernetes.io/projected/e86caf21-974c-444c-8b30-f553c77cd1be-kube-api-access-wmxrj\") pod \"e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.624018 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.623936 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:52.770070 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.770043 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv"] Jul 09 04:34:52.772051 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:34:52.772023 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode86caf21_974c_444c_8b30_f553c77cd1be.slice/crio-f46de3acdc36403cdcd17f3d59e7ae6019e6614a47cdc9fcce6a683f4a02c526 WatchSource:0}: Error finding container f46de3acdc36403cdcd17f3d59e7ae6019e6614a47cdc9fcce6a683f4a02c526: Status 404 returned error can't find the container with id f46de3acdc36403cdcd17f3d59e7ae6019e6614a47cdc9fcce6a683f4a02c526 Jul 09 04:34:52.774223 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.774192 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:34:52.774325 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.774274 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:34:52.774393 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:52.774341 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:34:53.459240 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:53.459201 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" event={"ID":"e86caf21-974c-444c-8b30-f553c77cd1be","Type":"ContainerStarted","Data":"27d7fbaa12720e7141c1c3e95e3cbafba52dc40e1b657f81870d2104a41369cd"} Jul 09 04:34:53.459435 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:53.459295 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" event={"ID":"e86caf21-974c-444c-8b30-f553c77cd1be","Type":"ContainerStarted","Data":"f46de3acdc36403cdcd17f3d59e7ae6019e6614a47cdc9fcce6a683f4a02c526"} Jul 09 04:34:53.480876 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:53.480800 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" podStartSLOduration=1.480783916 podStartE2EDuration="1.480783916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:34:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:34:53.479497394 +0000 UTC m=+2102.355415007" watchObservedRunningTime="2026-07-09 04:34:53.480783916 +0000 UTC m=+2102.356701503" Jul 09 04:34:53.624130 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:53.624094 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:53.628924 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:53.628899 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:54.463935 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:54.463908 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:34:54.465067 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:34:54.465046 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:35:10.506077 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:10.506030 2576 patch_prober.go:28] interesting pod/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 09 04:35:10.506509 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:10.506097 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" podUID="e86caf21-974c-444c-8b30-f553c77cd1be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 09 04:35:11.546073 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:11.546036 2576 generic.go:356] "Generic (PLEG): container finished" podID="e86caf21-974c-444c-8b30-f553c77cd1be" containerID="27d7fbaa12720e7141c1c3e95e3cbafba52dc40e1b657f81870d2104a41369cd" exitCode=0 Jul 09 04:35:11.546500 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:11.546109 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" event={"ID":"e86caf21-974c-444c-8b30-f553c77cd1be","Type":"ContainerDied","Data":"27d7fbaa12720e7141c1c3e95e3cbafba52dc40e1b657f81870d2104a41369cd"} Jul 09 04:35:11.546500 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:11.546482 2576 scope.go:117] "RemoveContainer" containerID="27d7fbaa12720e7141c1c3e95e3cbafba52dc40e1b657f81870d2104a41369cd" Jul 09 04:35:11.547507 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:11.547480 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:35:11.547567 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:11.547546 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:35:11.547617 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:11.547577 2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 09 04:35:12.553912 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:12.553868 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" event={"ID":"e86caf21-974c-444c-8b30-f553c77cd1be","Type":"ContainerStarted","Data":"40a031eecd6ba7cb117463fd4543f53976865c31a2542a872dc9ade4721d6f60"} Jul 09 04:35:12.625158 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:12.625111 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:35:12.625450 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:12.625422 2576 patch_prober.go:28] interesting pod/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 09 04:35:12.625522 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:12.625487 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" podUID="e86caf21-974c-444c-8b30-f553c77cd1be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 09 04:35:13.625446 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:13.625403 2576 patch_prober.go:28] interesting pod/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 09 04:35:13.625992 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:13.625472 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" podUID="e86caf21-974c-444c-8b30-f553c77cd1be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 09 04:35:14.638524 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:14.638474 2576 patch_prober.go:28] interesting pod/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 09 04:35:14.639074 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:14.638560 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" podUID="e86caf21-974c-444c-8b30-f553c77cd1be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 09 04:35:15.624700 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:15.624650 2576 patch_prober.go:28] interesting pod/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 09 04:35:15.624998 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:15.624760 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" podUID="e86caf21-974c-444c-8b30-f553c77cd1be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 09 04:35:16.628297 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:16.628263 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:35:16.628744 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:16.628518 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:35:16.629228 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:35:16.629210 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:39:51.669660 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:39:51.669630 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:39:51.673131 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:39:51.673106 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:39:51.676973 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:39:51.676952 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:39:51.679970 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:39:51.679951 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:41:43.238120 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:43.238036 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv"] Jul 09 04:41:43.238654 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:43.238271 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" podUID="e86caf21-974c-444c-8b30-f553c77cd1be" containerName="istio-proxy" containerID="cri-o://40a031eecd6ba7cb117463fd4543f53976865c31a2542a872dc9ade4721d6f60" gracePeriod=30 Jul 09 04:41:46.630776 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:46.630734 2576 patch_prober.go:28] interesting pod/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 09 04:41:46.631164 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:46.630800 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" podUID="e86caf21-974c-444c-8b30-f553c77cd1be" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 09 04:41:48.481334 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.481307 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:41:48.540037 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.539954 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e86caf21-974c-444c-8b30-f553c77cd1be-istio-podinfo\") pod \"e86caf21-974c-444c-8b30-f553c77cd1be\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " Jul 09 04:41:48.540037 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.540004 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e86caf21-974c-444c-8b30-f553c77cd1be-istiod-ca-cert\") pod \"e86caf21-974c-444c-8b30-f553c77cd1be\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " Jul 09 04:41:48.540037 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.540037 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-istio-envoy\") pod \"e86caf21-974c-444c-8b30-f553c77cd1be\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " Jul 09 04:41:48.540280 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.540074 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmxrj\" (UniqueName: \"kubernetes.io/projected/e86caf21-974c-444c-8b30-f553c77cd1be-kube-api-access-wmxrj\") pod \"e86caf21-974c-444c-8b30-f553c77cd1be\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " Jul 09 04:41:48.540280 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.540115 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-workload-certs\") pod \"e86caf21-974c-444c-8b30-f553c77cd1be\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " Jul 09 04:41:48.540280 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.540143 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-istio-data\") pod \"e86caf21-974c-444c-8b30-f553c77cd1be\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " Jul 09 04:41:48.540280 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.540180 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-credential-socket\") pod \"e86caf21-974c-444c-8b30-f553c77cd1be\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " Jul 09 04:41:48.540280 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.540202 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e86caf21-974c-444c-8b30-f553c77cd1be-istio-token\") pod \"e86caf21-974c-444c-8b30-f553c77cd1be\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " Jul 09 04:41:48.540280 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.540226 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-workload-socket\") pod \"e86caf21-974c-444c-8b30-f553c77cd1be\" (UID: \"e86caf21-974c-444c-8b30-f553c77cd1be\") " Jul 09 04:41:48.540637 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.540479 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e86caf21-974c-444c-8b30-f553c77cd1be" (UID: "e86caf21-974c-444c-8b30-f553c77cd1be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:41:48.540637 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.540540 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e86caf21-974c-444c-8b30-f553c77cd1be" (UID: "e86caf21-974c-444c-8b30-f553c77cd1be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:41:48.540637 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.540572 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e86caf21-974c-444c-8b30-f553c77cd1be-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e86caf21-974c-444c-8b30-f553c77cd1be" (UID: "e86caf21-974c-444c-8b30-f553c77cd1be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:41:48.540797 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.540675 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e86caf21-974c-444c-8b30-f553c77cd1be" (UID: "e86caf21-974c-444c-8b30-f553c77cd1be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:41:48.541085 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.541061 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-istio-data" (OuterVolumeSpecName: "istio-data") pod "e86caf21-974c-444c-8b30-f553c77cd1be" (UID: "e86caf21-974c-444c-8b30-f553c77cd1be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:41:48.542314 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.542291 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e86caf21-974c-444c-8b30-f553c77cd1be-kube-api-access-wmxrj" (OuterVolumeSpecName: "kube-api-access-wmxrj") pod "e86caf21-974c-444c-8b30-f553c77cd1be" (UID: "e86caf21-974c-444c-8b30-f553c77cd1be"). InnerVolumeSpecName "kube-api-access-wmxrj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:41:48.542453 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.542434 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e86caf21-974c-444c-8b30-f553c77cd1be-istio-token" (OuterVolumeSpecName: "istio-token") pod "e86caf21-974c-444c-8b30-f553c77cd1be" (UID: "e86caf21-974c-444c-8b30-f553c77cd1be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:41:48.542598 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.542577 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e86caf21-974c-444c-8b30-f553c77cd1be-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e86caf21-974c-444c-8b30-f553c77cd1be" (UID: "e86caf21-974c-444c-8b30-f553c77cd1be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 09 04:41:48.542692 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.542594 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e86caf21-974c-444c-8b30-f553c77cd1be" (UID: "e86caf21-974c-444c-8b30-f553c77cd1be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:41:48.641393 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.641341 2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-istio-envoy\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:41:48.641393 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.641390 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmxrj\" (UniqueName: \"kubernetes.io/projected/e86caf21-974c-444c-8b30-f553c77cd1be-kube-api-access-wmxrj\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:41:48.641393 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.641406 2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-workload-certs\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:41:48.641652 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.641419 2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-istio-data\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:41:48.641652 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.641430 2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-credential-socket\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:41:48.641652 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.641441 2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e86caf21-974c-444c-8b30-f553c77cd1be-istio-token\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:41:48.641652 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.641452 2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e86caf21-974c-444c-8b30-f553c77cd1be-workload-socket\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:41:48.641652 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.641462 2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e86caf21-974c-444c-8b30-f553c77cd1be-istio-podinfo\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:41:48.641652 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:48.641472 2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e86caf21-974c-444c-8b30-f553c77cd1be-istiod-ca-cert\") on node \"ip-10-0-137-39.ec2.internal\" DevicePath \"\"" Jul 09 04:41:49.330550 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:49.330517 2576 generic.go:356] "Generic (PLEG): container finished" podID="e86caf21-974c-444c-8b30-f553c77cd1be" containerID="40a031eecd6ba7cb117463fd4543f53976865c31a2542a872dc9ade4721d6f60" exitCode=0 Jul 09 04:41:49.330741 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:49.330567 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" Jul 09 04:41:49.330741 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:49.330570 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" event={"ID":"e86caf21-974c-444c-8b30-f553c77cd1be","Type":"ContainerDied","Data":"40a031eecd6ba7cb117463fd4543f53976865c31a2542a872dc9ade4721d6f60"} Jul 09 04:41:49.330741 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:49.330685 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv" event={"ID":"e86caf21-974c-444c-8b30-f553c77cd1be","Type":"ContainerDied","Data":"f46de3acdc36403cdcd17f3d59e7ae6019e6614a47cdc9fcce6a683f4a02c526"} Jul 09 04:41:49.330741 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:49.330714 2576 scope.go:117] "RemoveContainer" containerID="40a031eecd6ba7cb117463fd4543f53976865c31a2542a872dc9ade4721d6f60" Jul 09 04:41:49.339876 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:49.339834 2576 scope.go:117] "RemoveContainer" containerID="27d7fbaa12720e7141c1c3e95e3cbafba52dc40e1b657f81870d2104a41369cd" Jul 09 04:41:49.347438 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:49.347419 2576 scope.go:117] "RemoveContainer" containerID="40a031eecd6ba7cb117463fd4543f53976865c31a2542a872dc9ade4721d6f60" Jul 09 04:41:49.347727 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:41:49.347706 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40a031eecd6ba7cb117463fd4543f53976865c31a2542a872dc9ade4721d6f60\": container with ID starting with 40a031eecd6ba7cb117463fd4543f53976865c31a2542a872dc9ade4721d6f60 not found: ID does not exist" containerID="40a031eecd6ba7cb117463fd4543f53976865c31a2542a872dc9ade4721d6f60" Jul 09 04:41:49.347771 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:49.347736 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40a031eecd6ba7cb117463fd4543f53976865c31a2542a872dc9ade4721d6f60"} err="failed to get container status \"40a031eecd6ba7cb117463fd4543f53976865c31a2542a872dc9ade4721d6f60\": rpc error: code = NotFound desc = could not find container \"40a031eecd6ba7cb117463fd4543f53976865c31a2542a872dc9ade4721d6f60\": container with ID starting with 40a031eecd6ba7cb117463fd4543f53976865c31a2542a872dc9ade4721d6f60 not found: ID does not exist" Jul 09 04:41:49.347771 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:49.347752 2576 scope.go:117] "RemoveContainer" containerID="27d7fbaa12720e7141c1c3e95e3cbafba52dc40e1b657f81870d2104a41369cd" Jul 09 04:41:49.348066 ip-10-0-137-39 kubenswrapper[2576]: E0709 04:41:49.348048 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27d7fbaa12720e7141c1c3e95e3cbafba52dc40e1b657f81870d2104a41369cd\": container with ID starting with 27d7fbaa12720e7141c1c3e95e3cbafba52dc40e1b657f81870d2104a41369cd not found: ID does not exist" containerID="27d7fbaa12720e7141c1c3e95e3cbafba52dc40e1b657f81870d2104a41369cd" Jul 09 04:41:49.348134 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:49.348070 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27d7fbaa12720e7141c1c3e95e3cbafba52dc40e1b657f81870d2104a41369cd"} err="failed to get container status \"27d7fbaa12720e7141c1c3e95e3cbafba52dc40e1b657f81870d2104a41369cd\": rpc error: code = NotFound desc = could not find container \"27d7fbaa12720e7141c1c3e95e3cbafba52dc40e1b657f81870d2104a41369cd\": container with ID starting with 27d7fbaa12720e7141c1c3e95e3cbafba52dc40e1b657f81870d2104a41369cd not found: ID does not exist" Jul 09 04:41:49.353282 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:49.353260 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv"] Jul 09 04:41:49.358101 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:49.358078 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-7b94ae-openshift-default-5d88b469c5-2b8lv"] Jul 09 04:41:49.623465 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:49.623387 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e86caf21-974c-444c-8b30-f553c77cd1be" path="/var/lib/kubelet/pods/e86caf21-974c-444c-8b30-f553c77cd1be/volumes" Jul 09 04:41:56.288330 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:56.288299 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-df844f965-bdbfs_63d0fc88-2766-4c33-9a26-6f310339ecd2/copy-manifests/0.log" Jul 09 04:41:56.295223 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:56.295194 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-df844f965-bdbfs_63d0fc88-2766-4c33-9a26-6f310339ecd2/manager/0.log" Jul 09 04:41:56.716158 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:56.716130 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-557xq_5792dbfd-9d80-4de8-a04a-552a6bf8afd5/postgres/0.log" Jul 09 04:41:59.170245 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:59.170210 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-j4xcm_75bb459b-3eac-4cdd-be45-55d5d3affc0c/discovery/0.log" Jul 09 04:41:59.511693 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:41:59.511602 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-tqvrz_3ef6eb95-116d-41ea-b5a6-44159057e047/istio-proxy/1.log" Jul 09 04:42:00.201790 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:00.201744 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-bcs6d_51b3bac2-2bc9-4d7d-847a-7e32af5946c2/main/0.log" Jul 09 04:42:00.419328 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:00.419302 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-lklv4_d3a16cdc-993c-49c5-9395-5335ca9c586d/main/0.log" Jul 09 04:42:00.645301 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:00.645277 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-bm6fh_50d77141-63fe-4146-a629-50b044a5aa0f/main/0.log" Jul 09 04:42:07.936538 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:07.936508 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6mhqx_68ab2626-7f66-43db-b616-996085b15324/global-pull-secret-syncer/0.log" Jul 09 04:42:08.119007 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:08.118974 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-v8v6p_6d52430a-8806-461f-a26a-a4496add6d6f/konnectivity-agent/0.log" Jul 09 04:42:08.166605 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:08.166578 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-39.ec2.internal_cabda21fc7b0aa2caee43190e64a6bed/haproxy/0.log" Jul 09 04:42:15.545308 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:15.545205 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/1.log" Jul 09 04:42:15.688381 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:15.688352 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xrtbk_f0082ed4-ae5f-4d63-9096-12bf3a7903fa/cluster-monitoring-operator/0.log" Jul 09 04:42:15.817806 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:15.817723 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-ml5xr_34cd756f-1a12-45c1-9787-05fc3cfd17ed/monitoring-plugin/0.log" Jul 09 04:42:15.868894 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:15.868815 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cnbzz_d39f4d80-1a50-4766-998f-82afa4572f3a/node-exporter/0.log" Jul 09 04:42:15.894528 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:15.894498 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cnbzz_d39f4d80-1a50-4766-998f-82afa4572f3a/kube-rbac-proxy/0.log" Jul 09 04:42:15.917916 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:15.917893 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cnbzz_d39f4d80-1a50-4766-998f-82afa4572f3a/init-textfile/0.log" Jul 09 04:42:16.185898 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:16.185871 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_592a60a4-9e18-4329-9f82-ddd3fd72c8a1/prometheus/0.log" Jul 09 04:42:16.213917 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:16.213889 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_592a60a4-9e18-4329-9f82-ddd3fd72c8a1/config-reloader/0.log" Jul 09 04:42:16.238673 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:16.238644 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_592a60a4-9e18-4329-9f82-ddd3fd72c8a1/thanos-sidecar/0.log" Jul 09 04:42:16.263533 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:16.263508 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_592a60a4-9e18-4329-9f82-ddd3fd72c8a1/kube-rbac-proxy-web/0.log" Jul 09 04:42:16.288113 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:16.288088 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_592a60a4-9e18-4329-9f82-ddd3fd72c8a1/kube-rbac-proxy/0.log" Jul 09 04:42:16.310368 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:16.310337 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_592a60a4-9e18-4329-9f82-ddd3fd72c8a1/kube-rbac-proxy-thanos/0.log" Jul 09 04:42:16.334042 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:16.334011 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_592a60a4-9e18-4329-9f82-ddd3fd72c8a1/init-config-reloader/0.log" Jul 09 04:42:17.963804 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:17.963769 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc"] Jul 09 04:42:17.964204 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:17.964098 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e86caf21-974c-444c-8b30-f553c77cd1be" containerName="istio-proxy" Jul 09 04:42:17.964204 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:17.964110 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e86caf21-974c-444c-8b30-f553c77cd1be" containerName="istio-proxy" Jul 09 04:42:17.964204 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:17.964121 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e86caf21-974c-444c-8b30-f553c77cd1be" containerName="istio-proxy" Jul 09 04:42:17.964204 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:17.964127 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e86caf21-974c-444c-8b30-f553c77cd1be" containerName="istio-proxy" Jul 09 04:42:17.964204 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:17.964198 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e86caf21-974c-444c-8b30-f553c77cd1be" containerName="istio-proxy" Jul 09 04:42:17.964204 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:17.964207 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e86caf21-974c-444c-8b30-f553c77cd1be" containerName="istio-proxy" Jul 09 04:42:17.967491 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:17.967473 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:17.970346 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:17.970325 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v8skv\"/\"kube-root-ca.crt\"" Jul 09 04:42:17.971689 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:17.971673 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-v8skv\"/\"default-dockercfg-92sp8\"" Jul 09 04:42:17.971754 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:17.971673 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v8skv\"/\"openshift-service-ca.crt\"" Jul 09 04:42:17.977877 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:17.977833 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc"] Jul 09 04:42:18.108129 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.108092 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slmpn\" (UniqueName: \"kubernetes.io/projected/ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d-kube-api-access-slmpn\") pod \"perf-node-gather-daemonset-fghtc\" (UID: \"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:18.108303 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.108139 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d-podres\") pod \"perf-node-gather-daemonset-fghtc\" (UID: \"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:18.108303 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.108203 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d-sys\") pod \"perf-node-gather-daemonset-fghtc\" (UID: \"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:18.108303 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.108217 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d-lib-modules\") pod \"perf-node-gather-daemonset-fghtc\" (UID: \"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:18.108303 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.108243 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d-proc\") pod \"perf-node-gather-daemonset-fghtc\" (UID: \"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:18.209671 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.209626 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d-proc\") pod \"perf-node-gather-daemonset-fghtc\" (UID: \"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:18.209812 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.209686 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slmpn\" (UniqueName: \"kubernetes.io/projected/ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d-kube-api-access-slmpn\") pod \"perf-node-gather-daemonset-fghtc\" (UID: \"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:18.209812 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.209713 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d-podres\") pod \"perf-node-gather-daemonset-fghtc\" (UID: \"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:18.209812 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.209754 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d-sys\") pod \"perf-node-gather-daemonset-fghtc\" (UID: \"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:18.209812 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.209760 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d-proc\") pod \"perf-node-gather-daemonset-fghtc\" (UID: \"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:18.209812 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.209770 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d-lib-modules\") pod \"perf-node-gather-daemonset-fghtc\" (UID: \"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:18.210036 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.209912 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d-podres\") pod \"perf-node-gather-daemonset-fghtc\" (UID: \"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:18.210036 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.209926 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d-sys\") pod \"perf-node-gather-daemonset-fghtc\" (UID: \"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:18.210036 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.209937 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d-lib-modules\") pod \"perf-node-gather-daemonset-fghtc\" (UID: \"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:18.220306 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.220242 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slmpn\" (UniqueName: \"kubernetes.io/projected/ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d-kube-api-access-slmpn\") pod \"perf-node-gather-daemonset-fghtc\" (UID: \"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d\") " pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:18.278236 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.278199 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:18.408443 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.408412 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc"] Jul 09 04:42:18.411695 ip-10-0-137-39 kubenswrapper[2576]: W0709 04:42:18.411659 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podee3b41c7_45d0_4adb_a7a8_d82c82a03b7d.slice/crio-adfeae80292a8add42312fd35973f8507d19133f7b44226290adf411215b1ee4 WatchSource:0}: Error finding container adfeae80292a8add42312fd35973f8507d19133f7b44226290adf411215b1ee4: Status 404 returned error can't find the container with id adfeae80292a8add42312fd35973f8507d19133f7b44226290adf411215b1ee4 Jul 09 04:42:18.458168 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:18.458141 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" event={"ID":"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d","Type":"ContainerStarted","Data":"adfeae80292a8add42312fd35973f8507d19133f7b44226290adf411215b1ee4"} Jul 09 04:42:19.464174 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:19.464142 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" event={"ID":"ee3b41c7-45d0-4adb-a7a8-d82c82a03b7d","Type":"ContainerStarted","Data":"8444d5adc4e98e930bd7f7addab16da217e69fab9dacd1f8bcb9d14b5ee69b1d"} Jul 09 04:42:19.464609 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:19.464206 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:19.480416 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:19.480366 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" podStartSLOduration=2.480349762 podStartE2EDuration="2.480349762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:42:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:42:19.479669117 +0000 UTC m=+2548.355586706" watchObservedRunningTime="2026-07-09 04:42:19.480349762 +0000 UTC m=+2548.356267349" Jul 09 04:42:19.567527 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:19.567487 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-d8ppz_a6ba460a-0b9f-45d7-8bea-013f17b74d52/volume-data-source-validator/0.log" Jul 09 04:42:20.552493 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:20.552461 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gcm68_d595e0d2-6955-4f99-9531-47b6c1ce95a3/dns/0.log" Jul 09 04:42:20.576032 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:20.575999 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gcm68_d595e0d2-6955-4f99-9531-47b6c1ce95a3/kube-rbac-proxy/0.log" Jul 09 04:42:20.711319 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:20.711288 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-282rs_ef48e4fa-4394-40ac-8960-67646b4c0804/dns-node-resolver/0.log" Jul 09 04:42:21.300282 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:21.300248 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-55zxm_953f9350-02c8-4b0b-a2bb-28b305694ddb/node-ca/0.log" Jul 09 04:42:22.487890 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:22.487835 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-j4xcm_75bb459b-3eac-4cdd-be45-55d5d3affc0c/discovery/0.log" Jul 09 04:42:22.569514 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:22.569488 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-tqvrz_3ef6eb95-116d-41ea-b5a6-44159057e047/istio-proxy/0.log" Jul 09 04:42:22.626782 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:22.626753 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-tqvrz_3ef6eb95-116d-41ea-b5a6-44159057e047/istio-proxy/1.log" Jul 09 04:42:23.523555 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:23.523522 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p7z6b_53a12a15-f2c0-44ef-a267-6a540e093a33/serve-healthcheck-canary/0.log" Jul 09 04:42:24.035626 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:24.035586 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-dh5td_45f40f4a-afe4-4007-8f0a-d2df2dfa9914/insights-operator/0.log" Jul 09 04:42:24.036279 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:24.036255 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-dh5td_45f40f4a-afe4-4007-8f0a-d2df2dfa9914/insights-operator/1.log" Jul 09 04:42:24.164197 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:24.164166 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bdhx7_ee942429-8ea2-4e5c-b56a-6b4510d15878/kube-rbac-proxy/0.log" Jul 09 04:42:24.184563 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:24.184539 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bdhx7_ee942429-8ea2-4e5c-b56a-6b4510d15878/exporter/0.log" Jul 09 04:42:24.206232 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:24.206201 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bdhx7_ee942429-8ea2-4e5c-b56a-6b4510d15878/extractor/0.log" Jul 09 04:42:25.477950 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:25.477925 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-v8skv/perf-node-gather-daemonset-fghtc" Jul 09 04:42:27.129598 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:27.129570 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-df844f965-bdbfs_63d0fc88-2766-4c33-9a26-6f310339ecd2/manager/0.log" Jul 09 04:42:27.157148 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:27.157122 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-df844f965-bdbfs_63d0fc88-2766-4c33-9a26-6f310339ecd2/copy-manifests/0.log" Jul 09 04:42:28.081643 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:28.081608 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-j4xcm_75bb459b-3eac-4cdd-be45-55d5d3affc0c/discovery/0.log" Jul 09 04:42:28.139815 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:28.139780 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-tqvrz_3ef6eb95-116d-41ea-b5a6-44159057e047/istio-proxy/0.log" Jul 09 04:42:28.195764 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:28.195691 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-tqvrz_3ef6eb95-116d-41ea-b5a6-44159057e047/istio-proxy/1.log" Jul 09 04:42:28.505922 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:28.505826 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-765cd6b57c-mc2fx_8862ca9d-20f9-4c65-96e6-aebbe90d95be/manager/0.log" Jul 09 04:42:29.876517 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:29.876484 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-j4xcm_75bb459b-3eac-4cdd-be45-55d5d3affc0c/discovery/0.log" Jul 09 04:42:29.932695 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:29.932667 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-tqvrz_3ef6eb95-116d-41ea-b5a6-44159057e047/istio-proxy/0.log" Jul 09 04:42:29.994218 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:29.994183 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-tqvrz_3ef6eb95-116d-41ea-b5a6-44159057e047/istio-proxy/1.log" Jul 09 04:42:34.908671 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:34.908643 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rvsj4_a3544bd6-6306-4080-8a5f-d79d2dc9e69b/kube-multus-additional-cni-plugins/0.log" Jul 09 04:42:34.930785 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:34.930756 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rvsj4_a3544bd6-6306-4080-8a5f-d79d2dc9e69b/egress-router-binary-copy/0.log" Jul 09 04:42:34.951666 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:34.951637 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rvsj4_a3544bd6-6306-4080-8a5f-d79d2dc9e69b/cni-plugins/0.log" Jul 09 04:42:34.974386 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:34.974360 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rvsj4_a3544bd6-6306-4080-8a5f-d79d2dc9e69b/bond-cni-plugin/0.log" Jul 09 04:42:34.995159 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:34.995134 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rvsj4_a3544bd6-6306-4080-8a5f-d79d2dc9e69b/routeoverride-cni/0.log" Jul 09 04:42:35.017545 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:35.017514 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rvsj4_a3544bd6-6306-4080-8a5f-d79d2dc9e69b/whereabouts-cni-bincopy/0.log" Jul 09 04:42:35.036916 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:35.036879 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rvsj4_a3544bd6-6306-4080-8a5f-d79d2dc9e69b/whereabouts-cni/0.log" Jul 09 04:42:35.261975 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:35.261904 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-sptvx_1aae161d-ab19-425f-a426-685a31ec128d/kube-multus/0.log" Jul 09 04:42:35.282551 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:35.282522 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7zmf5_b13c1cbe-23d9-404e-b028-3efbf70d9fa0/network-metrics-daemon/0.log" Jul 09 04:42:35.300824 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:35.300789 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7zmf5_b13c1cbe-23d9-404e-b028-3efbf70d9fa0/kube-rbac-proxy/0.log" Jul 09 04:42:36.198915 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:36.198867 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-controller/0.log" Jul 09 04:42:36.215216 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:36.215191 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/1.log" Jul 09 04:42:36.215402 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:36.215222 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovn-acl-logging/0.log" Jul 09 04:42:36.233529 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:36.233505 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/kube-rbac-proxy-node/0.log" Jul 09 04:42:36.253967 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:36.253939 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/kube-rbac-proxy-ovn-metrics/0.log" Jul 09 04:42:36.270003 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:36.269975 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/northd/0.log" Jul 09 04:42:36.288770 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:36.288737 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/nbdb/0.log" Jul 09 04:42:36.308037 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:36.308010 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/sbdb/0.log" Jul 09 04:42:36.379155 ip-10-0-137-39 kubenswrapper[2576]: I0709 04:42:36.379124 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-974xm_70989833-87c2-4248-a174-c246e7205582/ovnkube-controller/0.log"